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PREFACE 


For  the  first  tifiie  in  1979  5  the  Research  and  Theory  Division  of  the 
Association  for  Educational  Communications  and  Technology  collected 
selected  research  papers  from  those  presented  at  the  national  convention 
in  New  Orleans,  LA  for  publication  in  this  "Proceedings"  document.  This 
publication  was  printed  in  limited  hardcopy  quantity^  but  Is  available 
through  the  Educational  ResQurces  Information  Clearinghouse  (ERIC)  system. 

REFEREEING  PROCESS:        All  papers  selected  for  presentation  at  the  Annual 

Convention  and  included  in  this  "Proceedings"  were 
subjected  to  a  blind  refereeing  process*    Proposals  were  submitted  by 
potential  presenters  to  Dr,  Bill  Winn  of  the  University  of  Calgary  who 
reMved  all  reference  to  author.    Proposals  were  then  submitted  to  reviewers 
for  evaluation.    Approximately  forty  percent  of  the  manuscripts  submitted 
for  consideration  were  selected  for  presentation  at  the  Convention  and  for 
publication  in  these  "Proceedings."    The  manuscripts  contained  in  this 
document  represent  some  of  the  most  current  thinking  in  the  educational 
communications  and  technology  field. 

While  each  paper  was  intended  to  stand  alone,  they  have  been  grouped 
according  to  a  general  category  for  ease  of  use.    Individual  authors 
should  be  questioned  concerning  their  ideas ,  methods,  or  conclusions. 
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Authors  •  FRANCIS  E,  CLARK 
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i  JAY  F,  ANGERT 
Texas  A&M  University 

Abstract  e  A  sunimarlzatlon  of  media  research  Inadequacies  remains  a  standard 
feature  of  media  literature  reviews,    A  Gestalt  solution  would  be  a  research 
modeU  applicable  to  field  studies  In  educational  Institutions,  which  accounts 
for  the  total  Instructional  setting.    It  appears  that  an  appropriate  research 
model  must  be  directly  related  to  the  psychological,  sociological,  and  physio- 
logical attributes  of  the  learner,  the  teacher,  the  task,  and  the  resources, 
which  collectively  result  in  an  instructional  environfnent.    The  objective 
form  (physiological)  and  subjective  meaning  (sociological)  of  any  learning 
task  must  yield  a  functional  distinctiveness  (psychological)  in  terms  of  the 
sensory  information  to  be  extracted  by  the  learner. 


RESEARCH  ON  MEDIA  -  WHERE  DO  WE  GO  FROM  HERE? 


FRANCIS  E.  CLARK 
Texas  ASM  University 
JAY  F,  ANGERT 
Texas  ASM  University 

A  summarization  of  media  research  inadequacies  remains  a  standard  fea- 
ture of  literature  reviews.    A  serious  reader  of  the  research  literature  for 
the  past  ten  years  will  encounter  numerous  depressing  appraisals  of  the 
limited  value  of  media  research  for  Improving  education.    Although  many  re- 
viewers have  decried  the  uncertain  quality  and  utility  of  media  research s  an 
exhaustive  litany  of  futility  is  unnecessary.    The  comments  which  follow  are 
sufficiently  illustrative, 

Ees^argh  DrnfioiGnaies 


In  1968,  Snow  and  Salomon  remarked  that  "virtually  nothing  is  known  .  * 
about  the  teaching  effectiveness  of  instructional  media"  (p.  341),    This  con-- 
elusion  was  based*  In  part,  on  their  observation  of  the  widespread  use  of 
experimental  designs  which  averaged  individual  learner  differencesi  although 
the  prifiie  Iniportance  of  these  differences  as  Independent  variables  had  been 
previously  noted  (Cronbach,  1957).    Comstock  (1975)  similarly  concluded  that 
the  utility  of  media  research  for  either  theory  or  practice  was  Inconsequen- 
tial.   In  addition^  research  with,  as  opposed  to  research  on  media,  has  been 
the  rule  rather  than  the  exception  (Salomons  1970).    Researchers  have  repeat- 
edly treated  a  given  medium  as  a  whole  entity,  as  in  comparison  studies  of 
film  versus  television.  In  an  attempt  to  support  the  premise  that  the  media 
could  indeed  teach-    Fleming  (1970)  recognized  that  such  gross  comparisons 


>^ielded  nieaningless  data  since  they  niasked  considerably  more  variability  than 
they  eKplained.    In  1977,  Schramm  described  this  macro  quality  as  perhaps  the 
most  regrettable  feature  of  the  long  V      of  instructional  media  experiments, 
Levie  and  Dickie  (1973)  suggested  th         earch  would  be  better  conceptualized 
by  specifying  media  variables  in  terni..  or  specific  attributes.    Finally,  Con- 
way (1970)  and  Dwyer  (1972)  have  identified  as  a  major  research  deficiency 
the  lack  of  logical  correlations  between  the  treatment  content  encountered  in 
many  studies  and  that  typical  of  actual  classroom  instruction.    Glaser  (1972) 
and  Salomon  and  Snow  (1970)  have  also  noted  the  pressing  need  for  "ecological 
validityj"  1*e.,  experlnientation  under  normal  instructional  conditions. 

Perhaps  the  most  brutally  frank  assessment  of  media  research  was  offered 
by  Hav/kridge  (1973) : 

The  fact  is  that  instructional  researchers  and  designers  have 
not  provided  even  the  foundations  for  constructing  strong  prac- 
tical procedures  for  selecting  media  appropriate  to  given 
learning  tasks.    If  there  has  been  British  work  in  this  area, 
I  have  been  unable  to  discover  it  ...  .    In  the  United  States, 
over  2O0O  madia  studies  have  not  yielded  the  answers  we  need. 
(P^  1) 

Taken  at  face  valusi  these  assessments  imply  a  great  deal  of  misdirected  energy, 
over  a  lengthy  time  spani  to  establish  a  data  base  of  questionable  value. 

ATI_ _a£  g  So t u tion 

The  widespread  recognition  that  media  research  had  failed  to  attend  to 
individual  learner  differences  prompted  repeated  calls  for  employing  the  meth- 
odology known  synonomously  as  Trait-Treatment  Interaction  (TTI),  or  Aptitude- 
Treatment  Interaction  (ATI),  hereinafter  referred  to  simply  as  ATI  (Allen, 
197U  Berliner  and  Cahen,  1973;  Cronbach,  1957;  Cronbach  and  Snow,  1977;  Di- 
Vesta,  1975;  Snow  and  Salomon,  1968;  Virag,  1976).    Within  this  context, 


aptitudes  or  traits  are  defined  broadl/  enough  to  include  the  ^syc^hological, 
sociological  and  physiological  characteri  sti  cs  of  learners.    C^onbacli  and 
Snow  (1977)  suggested  that  any  aspect  of  an  individual  which  my  be  tjseful 
in  predicting  instructional  responses  should  be  const dered  an  " aptitude-" 
Treatments  are  defined  in  a  similarly  broad  fashion  so  as  to  include  varia- 
tions among  most  experimentally  rtianipulable  aspects  of  the  teachingor  the 
environment. 

Interactions  may  be  defined  statisticany  as  regression  slopes  wvhich 
depart  from  parallel  ism.    A  di^sordinal  interactiort  suggests  th^t  dif^^erent 
treatments  are  differentially  superior  for  students  v^ho  are  at  different 
levels  of  a  particular  trait  or  aptitude.    An  ordj_nal   iriteraction,  however  ^ 
suggests  that  one  treatment  retains  its  superiority  over  an  alternate  ti^eat- 
ment  throughout  the  range  of  aptitudes  under  consideration-    Hov/ever^  this 
superiority  is  usually  more  pronounced  at  one  level  of  the  aptitude  than  at 
others  (Ott,  1977;  Cronbach  and  Snow,  1977), 

Adaptations  in  education  to  individual  differences  are  neither  new  nor 
<lifficult  to  find.    Considering  the  obvious  extent  to  which  the  educsitional 
community  has  accepted  Instru'^tlonal  and  prografnma tic  practices  geared  to 
individual  differences^  the  concerted  research  efforts  to  locate  eciucat ion- 
ally  relevant  ATI  surficlally  makes  eminent  good  sense.    It  is  intell  ectually 
difficult  to  deny  that  ATI's  exist,    To  do  so  is  tantamount  to  asserting  tliat 
the  instruction  which  works  best  for  one  group  of  students  is  therefore  best 
for  all  students  (Cronbach  and  Snow^  1977), 

Paradoxically^  it  1s  the  firfn  belief  in  hunian  individual  1ty  and  Instrijc- 
tional  diversity  which  has  so  complicated  ATI  research.    As  Cronbach  (1975] 
stated*  "Once  we  attend  to  interactions  s  we  enter  ^  hall  of  mirrors  iwhlch 
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extends  to  infinity  -  *       (p*  119).    The  greatest  difficulty  ATI  researchers 
have  faced  is  the  isolation  of  those  aptitudes  and  treatment  conditions i  from 
an  unknown  universe  of  differences ^  which  reliably  interact  with  particular 
Instructional  treatments  to  produce  predictable  learning  outcomes.  Consider- 
ing the  inrienslty  of  the  tasks  and  the  relative  infancy  of  the  technique^  it 
is  hardly  surprising  that  ATI  results  have  been  disappointing  (Brachtp  1970; 
Cronbach  and  Snow,  1977;  Dwyer,  1978;  Heidt,  1977;  Parkhurst,  1975).  Unde- 
niably, ATI  results  have  been  less  than  spectacular.    The  range  of  aptitudes 
and  treatnients  is  so  vast  that  researchers  have  had  a  veritable  field  day  in 
devising  researchable  combinations.    A  search  for  ATI's  calls  for  no  less  than 
a  survey  of  all  the  ways  in  which  Individuals  and  instructional  treatnients 
may  differ.    These  constructs  may  pair  up  to  form  virtually  limitless  ATI 
hypotheses  (Cronbach  and  SnoWs  1977).    The  result  has  been  a  bewildering  ar- 
ray of  studies  with  relatively  few  threads  of  commonaiity. 

In  the  face  of  such  diversity s  the  temptation  is  great  to  use  a  shotgun 
approach  in  searching  for  ATI's.    Salomon  (1971)  has  noted  a  tendency  on  the 
part  of  many  ATI  researchers  to  include  extremely  large  numbers  of  trait  mea- 
sures In  their  studies,  in  the  hope  of  discovering  some  Interactions,  Such 
interactions  5  even  when  founds  are  weak  in  their  explanatory  povver^  having 
arisen  atheoretically  and  from  an  Inadequate  conceptualization  of  the  traits. 
For  the  most  part*  this  broad-band  exploration  approach  has  not  been  success- 
ful, 

Attenipts  to  Integrate  the  fragmentary  ATI  results  have  met  with  only 
limited  success.    Allen  (1975)  concluded  that  generalizing  from  the  available 
results  was  virtually  iniposslble.    The  sifnilarity  of  his  comments  to  those  of 
Hawkridge  was  striking:        .  .  there  is  little  definitive  evidence  from  the 


aptitude-treatment  interaction  research  that  points  conclusively  to  the  em- 
ploj^ment  of  practices  that  might  guide  the  selection  of  the  more  general 
instructional  strategies ,  much  less  lead  to  the  design  of  specific  instruc- 
tional media"  (p.  139).    Dwyer  (1978)  and  Parkhiirst  (1975)  have  also  noted 
the  limited  usefulness  and  meaningful ness  of  ATI  research  to  date*    In  the 
summary  of  what  is  undoubtedly  the  seminal  work  for  research  on  interactions, 
Cronbach  and  Snow  (1977)  concluded  that  -'No  Aptitude  X  Treatment  interactions 
are  so  well  confirmed  that  they  can  be  used  directly  as  guides  to  instruction" 
(p^  492). 

Nunierous  methodological  problems  have  plagued  the  search  for  ATI  (Cron- 
bach and  Snowj  1977).    Quite  often,  however,  investigations  which  have  paid 
close  attention  to  acceptable  methodology  and  data  analysis  have  frequently 
paid  inadequate  attention  to  the  more  subtle^  but  equally  vital ^  manner  in 
which  the  particular  constructs  chosen  as  dependent  and  independent  variables 
complement  one  another.    Thus,  a  large  portion  of  the  media  research  short- 
comings has  stemmed  from  an  Inadequate  concGptual ization  of  pertinent  vari- 
ables.   Snow  (1970)  pointed  out  the  inappropriateness  of  the  majority  of 
constructs  In  differential  psychology  for  use  in  ATI  research,    Cronboch  and 
Snow  (1959)  previously  cited  the  need  for  new  conceptualizations  of  traits 
and  treatments.    However,  both  past  and  present  admonishments  have  largely 
gone  unheeded* 

The  ATI  research  literature  is  so  disparate  and  contradictory  that  re- 
viewers find  themselves  in  disagreement  over  its  proper  interpretation.  How 
IS  one  to  make  sense  out  of  a  body  of  research  which  fails  to  produce  inter- 
actions where  hypothesized,  produces  interactions  in  unanticipated  and  inex- 
plicable fashion,  and  which  may  or  may  not  replicate  interactions  in 


subsequent  studies?    Heidt  (1977)  stated  that  "To  prove  a  trait-treatment 
interaction,  it  Is  necessary  to  detect  a  disordlnal  Interaction  ,  . 
(p.  13),    Heidt  further  concluded  that  tna  ATI  results  are  so  Inconsistent 
that  general  sumnarlzation  1s  Inipossible.    Berliner  and  Cahen  (1973)5  how- 
ever, argued  that  ordinal  interactions  are  as  useful  in  ATI  research  as  dis- 
ordinal.    Contrasting  reviewer  techniques  have  further  muddied  the  waters. 
Of  the  ninety  studies  which  Bracht  examined,  only  five  were  adjudged  as 
giving  adequate  evidence  of  ATU  since  they  produced  disordinal  interactions 
Cronbach  and  Snow  (1977) ,  hov/ever,  regarded  Bracht *s  criteria  as  overly 
stringent  and,  1n  a  reexafninatlon  of  several  studies  dismissed  by  Bracht  as 
failing  to  show  ATI's*  found  disordinal  interactions* 

As  different  as  non-ATI  and  ATI  research  are  frorn  one  another  in  method 
ology  and  philosophy,  1t  is  interesting  to  note  that  many  of  the  criticisms 
leveled  against  ^on-ATl  research  are  equally  applicable  to  the  newer  method- 
ology.   To  the  extent  that  these  inadequacies  persist ,  confusion  will  still 
reign.    In  1970*  Shulman  warned  against  research  which  measured  aptitudes 
with  micrometers  but  envlronnients  with  divining  rods^  yet  critics  are  still 
decrying  the  unprof 1 tabil i ty  of  using  gross  aptitude  and  treatment  measures 
(Anderson,  Ball,  and  Murphy,  1975;  Cronbach  and  Snow,  1977;  Dwyer,  1978). 
According  to  Dwyer  (1  978)s  unrealistic  treatment  content  is  still  being  e?(- 
perimentally  varied  under  artificial  pedagogical  conditions  (Salomon  and 
Clark,  1977)*    Methodological  problems  (e^g.,  inadequate  sample  sizes  and 
data  analyses,  and  only  rare  replications),  continue  to  hamper  ATI  efforts 
(Berliner  and  Cahen,  1973;  Cronbach  and  Snow,  1977), 

These  difficulties  have  led  every  reviewer  of  ATI  literature  we  have 
encountered  to  paint  a  depressingly  familiar  picture  of  ineffectiveness. 


Alii  however^  have  been  loathe  to  suggest  abandoning  the  effort  and,  surpris- 
ingly, have  reached  somewhat  optimistic  conclusions  on  the  heels  of  pesslrnis- 
tic  reviews.    Perhaps  the  single  most  pervasive  shortcoming  of  ATI  fesearch 
efforts  that  we  have  detected  is  the  lack  of  incluslveness^  i.e.^  conective 
inattention  to  the  totality  of  the  learning  environment.    The  term  Aptitude- 
Treatment  Interaction  by  itself  denotes  an  overlj^  simplistic  two-dimensional 
conception  of  learning  environments  and  has  perhaps  engendered  a  delimited 
focus  among  some  researchers*    ATI  investigations  have  provided  a  forum  for 
researchers  to  promote  a  spectrum  of  variables  covering  1  earner ^  teacher^ 
and  treatment  characteristics;  environmental  or  situational  conditions;  and 
a  variety  of  resource  characteristics.    If  research  is  to  proceed  systemat- 
ically toward  usable  conclusions,  some  semblance  of  order  must  be  Imposed  on 
the  mass  of  ATI  hypotheses*    To  dates  research  has  been  conducted  from  each 
researcheT^S''Conception  of  fundamental  combinations  of  attributes.  However^ 
we  are  unaware  of  a  research  model  which  effeictively  relates  these  diverse 
attributes. 

The  literature  is  replete  with  suggestions.    Carpenter  (1972)  called  for 
a  blend  of  media  and  modes s  instructional  functions  and  objectives ,  content 
and  audience  characteristics*  and  learning  environtnents,  while  Clark  (1975) 
stressed  the  relationship  between  instructional  methods,  materials^  and  In- 
dividual aptitudes.    DiVesta  (1975)  suggested  corcentratlng  on  cognitive  pro- 
cessing variables,  whereas  Salomnn  (1976)  argued  that  a  presentation- s 
effectiveness  depended  on  a  match  of  mental  skills  activated  by  the  presen- 
tation's code  and  the  learning  task  requirements.    Finally,  Burns  (1976) 
suggested  a  blend  of  learner,  media  and  environniantal  variables^  while  Schramni 
(1977)  pointed  out  the  need  for  studies  of  the  content  of  instructional  media, 
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While  these  varied  recommendations  have  meritorious  features,  they  reflect 
both  an  obvious  lack  of  common  termfnology  and  a  vast  disagreement  on  a  cofn- 
prehensive  model  from  which  research  on  instructional  media  selection  should 
proceed. 

Since  the  empirical  data  have  not  provided  clear  research  directions,  we 
are  faced  with  a  body  of  literature  from  which  it  is  difficult  to  extract 
"general"  principles*    Thus^  we  will  need  to  extrapolate  from  a  broad  spec- 
trum of  models^  paradiginSi  classification  systems,  and  hierarchies.    In  this 
way  we  may  be  able  to  identify  the  common  denominators  of  the  current  litera- 
ture.   Any  resulting  research  modtl  would^  of  necessity^  be  eclectic  in  nature 

While  it  may  be  true  that  there  is  nothing  so  practical  as  a  good  theory, 
it  is  also  true  that  much  theorizing  has  little  relationship  with  practical- 
ity.   Instructional  researchers  would  do  well  to  adopt,  adapt,  and  apply  the 
eclectic  instructional  practices  of  '•successful  instructors  to  the  design  of 
Instructional  research.    Undeniably^  a  considerable  amount  of  classroom  In- 
structions devoid  of  expeninental  coiitrols  or  conStraincSj  frequently  produces 
learning  of  practical  significance.    One  possible  explanation  may  lie  in  the 
holistic  approach  which  is  characteristic  of  the  "effective"  instructori  but 
whichs  philosophical ly,  is  worlds  apart  frorn  most  resevrcii  efforts.  Intui- 
tively^ many  instructors  manage  to  derive  an  optimal  blend  of  personal  style, 
learner  and  resource  characteristics,  and  task  requirsnients  through  a  consid- 
eration of  psychological  1  sociological,  and  physiological  factors. 

All  too  frequently,  however,  the  researcher  operates  from  a  narrov/ly 
circumscribed  perspective.    This  tunnel  vision  tendency  leads  to  narrowly 


conceived  rtsearch  hypothests.    These  hypotheses ^  in  turn,  engender  explana- 
tions of  data  in  terms  which  either  sustain  or  modify  the  philosophic  bent 
of  a  particular  researcher.    Rarely  are  phnDSOphic  lines  crossed  to  acknow- 
ledge equally  valid  or  potentially  superior  explanations  of  observed  learning 
outcomes*    This  omission  is  true  both  across  and  within  disciplines.  The 
field  of  educational  technology  serves  as  an  appropriate  example. 

We  assert  that  past  media  research  has  been  philosophically  dichotomous ^ 
arising  eliher  froiTi  practitioner  or  researcher  concerns.    Becker  (1977)  de- 
scribed how  such  artificial  distinctions  have  excluded  consideration  of  many 
important  variables*    Not  surprisingly,  the  various  media  selection  models 
have  closely  paralleled  these  media  research  directions,    Mieike  (1973)  re- 
ferred to  this  separation  as  the  distinction  between  administrative  research 
and  basic  research,  and  Clark  (1975)  extended  this  distinction  to  most  media 
taxonomies  * 

An  Ealsa^ig J^^cist  foi^  Researah  and  Ineti^wtigft 

To  a  Urg§  extent ,  commitment  to  preferred  statistical  niethodology  has 
also  dictated  rGsgarch  directions.    Usually g  investigators  hawe  dealt  with 
one  or  two  instructional  variables  at  a  time,  either  fn  a  search  for  main 
effects  or  for  interactions*    To  this  ends  researchers  have  used  regression 
analysis  and  other  sophisticated  statistical  techniques  to  analyze  fairly 
unsophisticated  subjective  measures  of  attributes  or  traits.    For  these  rea- 
sonSs  statistical  trends  have  failed  to  produce  the  consistency  needed  for 
the  development  of  an  instructional  modeK 

Considering  the  unproductive  history  of  research  on  instruction.  It  seems 
appropriate  to  step  back  and  take  a  second  look  at  the  diverse  research 


directions  which  have  beeni  to  a  great  extents  independent  of  one  another* 
While  we  recognize  that  research  frequently  transcends  artificial  boundaries , 
we  nevertheless  submit  that  most  research  on  Instruction  may  be  categorized 
loosely  into  three  major  areas.    We  feel  that  these  areas  closely  parallel 
the  considerations  of  the  effective  Instructor  described  earlier, 

The  first  area  may  be  termed  functional  and/or  differential  psychologi- 
Ml  research*    This  area  deals  primarily  with  intellectual  abilities,  as  well 
as^  the  relationships  among  itimulij  mediating  covert  behaviors  ^  and  observable 
overt  responses*    More  precisely ^  researchers  within  this  domain  of  research 
usually  begin  with  a  psychological  theory  and  then  proceed  to  validate  the 
logically  derived  statements ^  in  the  form  of  constructs ^  through  schemes  for 
organizing  data  for  quantitative  analysis.    The  following  schemes  are  usually 
thought  to  be  synonomous:    theory^  model,  paradigm*  analogy^  structure,  hier- 
archy, and  system.    E.  L.  Thorndike,  J.  P.  Guilford,  R.  M.  Gagne,  B,  S.  Bloom^ 
D.  P.  Ausubel 5  L.  S.  BrIggSj  G,  A.  Salomons  L.  J,  Cronbach,       L.  Thurstone^ 
R.  IL       TraverS,  R.       Clarke  B.  F.  Skinner,  C.  E.  Osyuud^  dnd  G,  L;  Gropper 
are  some  of  the  researchers  who  have  followed  this  line  of  research, 

A  second  area  deals  with  observational  and/or  sociological  research, 
f^esearchers  in  this  area  acknowledge  the  notion  that  individuals  can  learn  to 
perform  some  physical  and  social  tasks  by  Imitating  the  overt  behaviors  of  a 
"model,"    This  research  area  includes  the  hunianistiCi  cultural,  ethic*  ethnic* 
ego  and  consistency  needs  of  individuals  when  they  are  alone  or  1n  groups. 
This  area  also  encompasses  instructional  cognitive  styles  which  may  be  cul= 
tural  and  social  preference  systems  acquired  and  supplemented  during  schooling 
(Heidts  1977),    Some  of  the  proponents  of  the  sociological  approach  to  in- 
struction include  the  following:    A,  Bandura,  G.  F,  Kuder,  E,  K,  Strong^  G. 
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Allport',  H.  A.  Witkin,  N.  Flanders,  W.  F.  Seibert,  R,  E,  Snow,  R.  Sears, 
A,  H,  Maslow,  C-  Buhler,  and  E,  H*  Erikson. 

The  third  area  is  defined  as  physiological  research.    Within  this  area 
lies  the  subjective  research  on  perceptions  on  the  form  and  structure  of  sen-' 
sory  inessages,  and  on  the  constant  interaction  between  the  person  and  the 
environment.    The  biological  bases  of  khowledge  contain  the  roots  of  uhis 
major  area.    Research  in  this  domain  has  dealt  with  the  developmental  charac- 
teristics of  Individusls  as  they  Interact,  through  the  sensory  channyis^  with 
the  instructional  environment.    S.  H,  Bartley,  C.  B.  DeSoto,  J.  J.  Gibson, 
L.  Fleniing,  J.  Piaget,  A.  Gesen  ,  R.  J.  Havlghursts  H.  Werner^  M.  Montessori, 
D*  Durkins  V.  Lowenfeld,  F,  M.  Dwyers  and  A.  A.  Lumsdaine  are  some  of  the  re- 
searchers who  have  contributed  to  this  area  of  study. 

It  appears  that  an  appropriate  research  model  must  be  directly  related 
to  th^  psychological  5  sociological ,  and  physiological  attributes  of  the  learn- 
er, the  teacher,  the  task,  and  the  resources^  which  collectively  result  in  an 
in.^tructional  environment,    A  Gestalt  solution  would  be  a  research  models 
applicable  to  field  studies  in  educational  institutions,  which  accounts  for 
the  total  instructional  setting.    If  learning  is  the  ultimate  product  of  the 
instructional  environment,  then  the  Instructional  environment  is  the  product 
of  the  interaction  within  *  between,  and  among  the  teacher,  the  learner,  the 
task,  and  the  resources.    One  of  the  problems  involved  in  a  discussion  of  our 
instructional  model  is  that  the  variables  are  not,  unfortunately ^  as  mutuany 
exclusive  as  we  would  like  for  them  to  be.    We  are  viewing  instruction  as  a 
dynamic  process  in  which  the  variables  of  instruction  play  an  integral  but 
subordinate  part.    Of  greatest  importance  are  the  unique  psychological,  socio- 
logical ^  atnd  physiological  relationships  within,  between^  and  among  the 


variables.    We  maintain  that  relationships  between  stimuli  and  responses  are 
best  predicted  from  information  about  the  intermediary  procisses  that  occur 
within  the  individual.    It  is  not  unreasonable  to  suppose  that  the  learner 
has  developed  general  dispositions  for  processing  stimulation  based  upon  the 
daily  activities  associated  with  communicating^  perceiving  temporal  and  spa-- 
tial  relations^  and  problem  solving.    The  learner  must  adapt  himself  to  the 
learning  environment  in  order,  ultimately,  to  learn.    Our  model,  then,  is 
deduced  from  the  psychological,  sociological,  and  physiological  makeup  of 
the  learner  and  his  surroundings.    To  use  Dale's  (1969)  "Cone  of  Experiences" 
as  a  simplistic  example,  if  the  learning  environment  is  too  -'concrete,''  the 
learner  will  be  under  stimulated;  if  overly  "abstracts"  the  learner  will  be 
overwhelmed*    In  either  case,  it  is  quite  likely  that  the  learner  will  not 
reach  the  objective  of  the  learning  task*    The  implication  sought  here  Is 
that,  in  order  to  provide  a  functional  relationship  within,  between,  and 
among  the  variables  of  instruction^  communications  problems  involving  syn- 
tactics (interrelations  of  signs),  semantics  (meanings  attributed  to  signs), 
and  pragmatics  (human  reactions  to  signs)  must  be  minimal.    Hence,  the  ob- 
jective form  (physiological)  and  subjective  meaning  (sociological)  of  the 
learning  task  must  yield  a  functional  distinctiveness  (psychological)  in 
terms  of  the  sensory  information  to  be  extracted  by  the  learner. 

Internal  consistency  within  each  variable  of  instruction  is  achieved 
only  when  the  psychological,  sociological,  and  physiological  attributes  are 
encoded  and  decoded  in  harmony,    VJhile  it  may  be  unrealistic  to  attempt  to 
reduce  the  complexities  of  human  nature  to  purely  numerical  terms,  it  seems 
worth  emphasizing  that,  if  viewed  in  the  manner  v/e  have  described,  there  is 
a  potential  of  seven  interactions  within  each  variable  of  Instruction  for 


any  given  learning  task.    Carried  further,  the  potential  interactions  among 
the  variables  of  an  instructional  environnient,  coniprised  of  the  learner,  the 
teacher,  the  task,  and  the  resources ^  may  be  derived ^  in  conservative  iTiathe» 
matical  terms ?  as  approximately  three  million  total  Interactions  (HoaU  Port 
and  Stonej  1971),    Since  these  permutations  are  based  upon  one  learner,  rather 
than  a  class  or  cell  of  twenty  or  more  learners,  the  iniplications  are  explic- 
it.   At  any  rate,  these  concomitant  considerations  serve  to  illustrate  the 
complexity  of  any  given  instructional  environment.    The  complexity  is  rnagnl- 
fied  still  further  by  the  fact  that  perceived  dynamic  internal  and  external 
attributes  1  that  seem  to  be  uniquely  associated  with  the  variables  of  the 
learning  environment ^  are  not  always  accurate  reflections  of  the  actual  unique 
attributes.    Over  time,  and  through  controned  research^  the  actual  unique 
attributes  for  different  learning  environments^  may  mainfest  themselves  as  a 
subset  of  the  perceived  attributes,    There  Is  stills  however,  an  additional 
complication,  i.e.,  that  either  perceived  or. actual  attributes  may  evolve^ 
change^  or  disappear  during  the  course  of  instruction  or  experimentation 
either  due  to  maturation  or  due  to  interaction  with  other  elements  of  the 
instructional  environment.    In  the  light  of  these  considerations,  it  seems 
particularly  apt  to  note  that  Cronbach  (1975)  said       .  .  the  line  of  inves- 
tigation I  advocated  in  1957  no  longer  seems  sufficient.    Interactions  are 
not  confined  to  the  first  order;  the  dimensions  of  the  situation  and  of  the 
person  enter  Into  complex  Interactions"  (p.  US)* 

l/her&  Do  y&  Go  From  Here? 

It  would  seefTi  that  the  need  exists  for  an  Instructional  model  that  in- 
corporates the  past,  present,  and  future  researchable  directions.    This  1s 
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not  to  say  that  future  researchers  will  agree  that  the  postulated  relationships 
in  vogue  today  are  researchable  tomorrow.    Nonetheless,  it  may  be  possible  to 
use  the  model  discussed  hereinbefore  as  a  foundation  or  datum  plane  upon  which 
a  logical  rationale  could  be  based  for  future  meta-analyses.    At  the  same  time 5 
psychological,  sociological  ^  and  physiological  measuring  Instrunients  and/or 
inventories  with  veridical  comparisons  should  be  Identified  and/or  developed 
in  an  attempt  to  differentiate  the  unique  attributes  from  the  common  and  the 
static  attributes  from'  the  fiynamic.    As  a  result^  functional  eKperimfintal 
research  investigating  the  Interrelationships  of  the  variables  of  instruction 
will,  in  the  future^  be  equipped  to  employ  realistic  rational  controls  so  that 
experimenters  may  more  reliably  explain  that  which  actually  happened. 

Most  contemporary  researchers  would  agree  that  we  need  to  know  more  about 
the  physical  and  psychological  attributes  of  resources,    Heldt  (1977)  suggested 
that  the  unique  psychological  attributes  of  resources  may  be  a  product  of  the 
physical  attributes  for  specific  learning  experiences*    We  submit  that  a  simi- 
lar subset  of  unique  sociological  attributes  should  also  be  specified  since 
they  Influence,  and  are  influenced  by^  the  unique  psychological  and  physiologi- 
cal attributes.    To  extend  this  ideas  the  subsets  should  also  be  comparable  to 
all  of  the  variables  of  instructions  and  not  just  to  the  resources. 

In  conclusion,  it  seems  reasonable  to  believe  that  a  logically  deduced 
amalgamation  of  all  research  in  the  behavioral  sciences  could  result  In  one 
or  more  axiomatic  theories  for  Instruction.    Once  derived*  selected  constitu- 
ent attributes  could  be  held  constant  whiles  at  the  same  time,  systematically 
varying  others*    Until  this  Is  accompl ished,  research  on  Instructional  media 
will  remain  omnibuSs  composed  of  complex  and  multivariate  aspects  of  what 
might  be  termed  "impulsive  reckoning*" 
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Reseafch  Into  Practice 


(Presentation  Abstract) 

RESEARCH  INTERPRETATION:    A  JUSTIFICATION  FOR  MEDIA 

This  paper  will  attempt  to  explain  the  apparent  inconsistencies 
in  media  research  findings  by  identifying  apparenv  research  weaknesses  which 
complicate  data  interpretation  and  frustrate  attempts  to  derive  broad 
generalizations  useful  to  practitioners  in  their  classroom  use  of  visual 
media.    The  presentation  will  emphasize  the  interrelationships  which  exists 
among  the  different  types  of  visual  materials,  different  types  of  educational 
objectives,  different  methods  of  presentation,  different  cueing  techniques, 
etc.    Studies  describing  the  results  obtained  from  evaluations  involving 
television,  slide/audiotape,  programmed  instruction,  and  textbook  formats  will 
be  reviewed. 
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RESEARCH  INTERPRETATION i    A  JUSTIFICATION  FOR  MEDIA 


Education  is  currently  eKperlencing  financial  crises  which  are  necessitating 
that  virtually  everyone  justify  the  financial  outlay  made  for  his  position, 
equipments  and  materials.    At    all  levels  instructional  media  specialists  and 
users  of  educational  technology  are  being  challenged  to  defend  their  usefulness 
in  the  teaching-learning  process.  This  is  not  always  easy  to  do  because  of  the 
contradictory  nature    of   empirical      findings  which  have  resulted  from  media 
related  research  studies.    For  example^  1f  one  were  to  survey  the  available 
experimental  media  research,  a  number  of  studies  would  be  found  in  which  instruc- 
tion utilizing  media  was  significantly  more  effective  than  conventional  instruction 
Some  studies  would  also  be  found  in  which  the  mediated  Instruction  was  found  to  be 
significantly  less  effective  than  conventional  instruction,  and  others  would  be 
found  In  which  no  differences  were  found  to  exist  between  the  mediated  and 
conventional  instruction.    To  further  complicate  media  justification,  when  signifi- 
cant differences  are  obtained  In  empirical  studieSt  the  results  are  seldom  in 
agreement  with  other  research  findings  investigating  similar  problems  (Davis, 
1962;  Wendt  &  Butts,  1962;  Stickell,  1963;  Reid  &  MacLennan,  1967;  Chu  &  Schramm, 
1967;  Edllng,  1968) •    Opponents  of  media  and  technology  are  quick  to  cite  these 
contradictory  findings  as  an  indication  of  Ineffectiveness* 

Since  it  has  been  documented  that  properly  designed  visual  materials  can 
significantly  Improve  student  achiev  ement  (VanderMeer,  1950b;  Kopstein  &  Roshal , 
1954;  Treichler,  1957'.  Gropper,  1962;  Dwyer,  1972  ,  1978),  it  is  imperative  that 
individuals  charged  with  the  responsibility  of  justifying  media  utilization  be 
able  to  detect  those  deficiencies  in  media  research  which  contribute  to  its 
failure  to  generate  consistent  findings  of  superiority*    After  deficiencies  are 
identified,  contradictory  findings  can  be  explained  and  the  use  of  media  justified. 


Following  1s  a  list  of  some  of  the  most  common  criticisms  associated  with 
media  research  which  tend  to  complicate  data  interpretation  and  frustrate  any 
attempts  to  derive  broad  generalizations  useful  to    practitioners  in  the  class* 
room  (Dwyer,  1978  pp.  39-41): 

1,  Many  studies  have  obvious  weaknesses  in  experimental  design, 
i,e.,  lack  of  randomization  of  students,  vagaries  in 
sampling,  inadequate  numbers  of  students  in  treatment 
groups,  lack  of  tests  of  significance  and  probability 
statements,  absence  of  any  control  factors. 

2,  A  considerable  number  of  media  related  studies  which  have 
been  reported  are  without  any  hypotheses  or  predictions 
based  on  theory* 

3*    The  content  material  being  presented  experimentally  in 

media  related  studies  has  been  restricted  in  that  it  is  far 
removed  from  that  which  is  currently  being  taught  in 
schools  (nonsense  syllables,  digits^  letters,  etc); 
consequently^  the  results  have  had  little  practical 
significance  to  educators  involved  in  applications  in 
the  classroom, 

4,  The  difficulty  and  meaningful ness  of  the  content  material 
(in  terms  of  the  kinds  of  educational  tasks  to  be 
achieved  by  the  students)  used  in  the  experimental 
treatments  has  not  been  specified  precisely. 

5.  In  many  media  related  studies  content  to  be  taught  has 

not  been  pretested  to  determine  where  visualization  of  the 

the  content  is  appropriate;  in  other  words,  a  considerable 

amount  of  visualization  used  in  media  related  experiments 

has  not  been  specifically  designed  to  complement  the  content 

■ 

material  to  be  presented. 


In  media  related  research  the  relationship  that  exists 
between  the  content  information  in  each  channel  (visual, 
oral,  print)  to  the  Information  in  the  other  channel (s) 
needs  to  be  specified  precisely. 

The  method  of  presenting  the  visualization  has  not  been 
described  precisely. 

The  precise  purposes  of  visualization  in  many  mediated 
studies  has  not  been  stated. 

The  amount  of  realistic  det&il  (line  drawings,  detailed, 
line  drawings,  photographs)  contained  in  visualization 
designed  to  complement  oral/print  instruction  in  media  researc 
has  not  been  described  precisely  so  that  data  from  different 
studies  can  be  compared* 

Mediated  instruction  normally  is  evaluated  via  printed 
criterion  Items,    To  properly  assess  the  instructional 
contribution  of  visualization,  a  significant  proportion 
of  the  evaluation  battery  should  reflect  all  the  channels 
(visual,  print,  oral,  etc  J  used  by  the  learner  to  acquire 
the  information* 

Many  of  the  criterion  tests  currently  being  used  to  measure 
achievement  are  global  in  nature  attempting  to  measure  the 
students'  total  learnings  rather  than  being  designed  to 
measure  media's  contribution  to  students'  achievement  of 
specific  kinds  of  educational  objectives.    In  using  the  global 
criterion  test  to  measure  student  achievement  of  different 
types  of  objectives,  the  variances  are  pooled  (for  the 
different  objectives^  thereby  concealing  any  effects  that 
media  may  have  in  fadliatlng  student  achievement  of  specific 
educational  objectives* 


Many  of  the  criterion  measures  used> to  assess  student 
information  acquisition  may  be  invalid  or  unreliable-- 
reliability  and  validity  coefficients  are  very  infrequently 
reported  in  media  related  research. 

The  type  of  test  items  used  to  measure  student  information 

acquisition  has  not  been  adequately  specified.    This  is 

important  because  item  format  can  influence  the  student's 

level  of  achievement;  i.e.,  a  fill-in  type  test  in  which 

the  student  is  required  to  provide  the  correct  word  is 

more  difficult  than  a  recognition  type  multiple-choice 

type  test  in  which  the  student  Is  required  to  select  the 

correct  response  from  an  array  of  possible  aUernatives. 

Researchers  have  failed  to  specify  the  time-span  (seconds, 

minutes,  hours,  etc  J  between  the  presentation  of  the 

information  to  the  students  and  the  testing. 

The  amount  of  time  students  have  been  permitted  to  view 

and  interact  with  the  Instructional  presentations  has  not 

been  adequately  controlled  and/or  reported. 

Media  researchers  in  preparing  their  research  for 

publication  have  omitted  essential  characteristics  of 

their  studies  which  prevents  exact  replication,  i.e.,  total  time 

of  treatment,  age  or  grade  level  of  students,  mean  intelligence 

of  students,  prior  knowledge  in  the  content  area,  etc. 

Very  few  of  the  media  related  studies  have  been  replicated 

to  establish  confidence  in  the  results. 

The  diversity  of  Interests  and  a  lack  of  common  usable 

terminology  used  by  media  researchers  has  compounded  the 

misinterpretation  of  many  media  related  studies  and  restricted 

their  general izability. 

do 


19,    To  a  certain  extent  the  results  of  experiments  have  been 
determined  by  the  statistical  techniques  used--a  liberal 
test  providing  one  interpretation  of  the  resultsi  a 
conservative  statistic  another. 
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Abstract 

Behavioral  objectives  provide  the  basis  for  systematic  planning  of 
instruction.    This  systematic  approach  enables  the  designer  to  work  more 
effectively  and  it  enables  the  learner  to  understand  what  is  expected  upon 
completion  of  the  loarning  experience.    There  aret  however,  a  wide  range  of 
views  concerning  the  advantages  of  behavioral  objectives.    The  purpose  of  this 
paper  is  to  present  a  critical  review  of  the  literature  on  behavioral  objectives. 
This  paper  Is  divided  into  four  major  areas:    (1)  definition  of  a  behavioral 
objective;  (2)  the  function  of  behavioral  objectives;  (3)  a  consistent  acceptable 
format  for  constructing  behavioral  objectives^  and  (4)  the  pros  and  cons  of 
behavioral  objectives. 

The  literature  review  reveals  that  current  findings  on  the  effects  of 
instructional  objectives  provide  no  conclusive  or  consistent  data  on  the 
relationship  between  the  use  of  objectives  and  student  learning*    Consequently , 
there  is  a  need  to  assess  the  behavioral  objectives  movement t  to  identify 
strGogths  and  weaknesses,  and  to  suggest  areas  in  which  research  is  needed* 
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Introduction 

Behavioral  objectives  have  been  central  to  the  concept  of  Instructional 
systems  development.    They  have  now  been  incorporated  into  the  designing  of 
curriculum.    They  provide  the  basis  for  planning  instruction.    They  have  been 
used  to  tell  learners  what  is  expected  of  them  upon  completion  of  the  learning 
experience.    There  iSp  hov^everp  a  wide  range  of  views  concerning  the  advantages 
of  behavtoral  objectivesi  as  well  as  niany  varying  opinions  as  to  the  technical 
aspects  of  how  and  for  what  purpose  they  should  be  used*    Behavioral  objectives 
provide  a  point  of  departure  for  a  thoroughgoing  attempt  to  improve  instruction 
By  precisely  stating  in  behavioral  terms  what  the  student  should  be  able  to 
do  after  the  learning  eKperiences  the  designer  hopes  to  reduce  any  gaps 
between  the  desired  outcQmes  of  education  and  the  Intentions  of  the  instructor. 
This  approach  has  been  criticized  from  both  curriculum  specialists  and 
educational  technologists*    This  paper  is  a  critical  review  of  the  literature 
on  behavioral  objectives;  it  1s  divided  Into  four  parts:    (1)  determining  a 
consistent  definition,  (2)  a  consistent  acceptable  fonriat  used  for  writing 
behavioral  objectiveSs  (3)  the  function  of  behavioral  objectives^  and  (4)  the 
cases  for  and  against  the  use  of  behavioral  objectives, 

I    Defining  Behavioral  Objectives 

At  fiHt  it  may  seem  that  defining  objectives  is  really  not  a  difficult 
task*    However,  educators  experienced  with  curriculuni  development j  course 
development^  and  those  who  have  tried  to  develop  procedures  for  evaluating 
students  will  attest  to  the  fact  that  it  is.    Palmer  (1974)  feels  that  most 
educators  make  the  task  of  defining  behavioral  objectives  far  too  complicated. 


This  section  of  the  paper  will  be  divided  into  five  parts:    (1)  Defining 
Behavioral  Objectives  in  Relation  to  Terminal  Behaviors  (2)  Defining 
Behavioral  Objectives  in  Rtlation  to  Subject  Matter,  (3)  Operational  ism  and 
Behavioral  ObjectlveSi  (4)  Opposing  View  Points  Concerning  the  Definition, 
and  (5)  Research  Related  to  the  Development  of  a  Consistent  Operational 
Defini tion. 

Defining  Behavioral  Objectives  in  Relation  to  Terminal  Behavior 

Many  educators  have  defined  the  term  behavtoral  objective,  Lindvall 
(1964)  states  that  the  process  of  developing  behavioral  objectives  is  basically 
one  of       facilitaving  communicati on.    This  is  accomplished  by  choosing 
precise  v/ords  and  statements  so  that  there  is  a  clear  and  exact  meaning  for 
those  reading  the  objective*    Pophani  (1969)  v^rites  that  whether  these  state- 
ments are  referred  to  as  objectives ^  alms,  goals ^  intents,  or  outcomes  is 
relatively  unimportant.    Whatever  synonym  Is  used^  a  behavioral  objective 
should  refer  to  an  intended  change  v/hlch  one  wishes  to  bring  about  in  a 
learner.    Bloom  (1956)  defines  objectives  as  being  specific  formulas  that  the 
educative  process  uses  to  change  student  behavior.    Mager's  (1962)  definition 
of  behavioral  objective  has  probably  influenced  more  educators  than  any  other 
definition:    an  objective  is  a  statement  describing  a  proposed  change  in  a 
learner;  It  specifies  what  the  learner  will  be  like  when  he  has  successfully 
completed  a  learning  experience.    For  example,  an  objective  written  for  a 
5th  grade  science  class  using  Mager*s  3  characteristics  of  a  well  stated 
objective  would  look  like  the  following:    Given  a  battery,  light  bulb,  socket, 
and  pieces  of  wire,  the  student  will  be  able  to  demonstrate  the  making  of  an 
electric  circuit  by  connecting  wires  to  battery  and  socket  and  testing  the 
lighting  of  the  bulb, 
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Defining  Behavioral  Objectives  in  Relation  to  Subject  Matter 

A  rnajor  consideration  when  one  is  defining  behavioral  objectives  1s  that 
of  determining  what  is  to  be  learned  by  the  learner.    An  educational  objective 
has  been  described  as  one  in  which  the  learner's  behavior  is  clearly  and 
precisely  specified  in  relation  to  the  subject  matter  with  which  the  learner 
is  expected  to  deal.    That  is^  the  objective  must  specify  not  only  the  learner's 
terminal  behavior,  but  also  the  part^'cular  aspects  of  the  subject  matter  to 
which  the  learner  must  address  himself  in  order  that  learning  may  occur, 
Gagne*  and  Briggs  (1974)  state  that  the  first  step  In  defining  objectives  is  to 
identify  the  purpose  of  the  course.    This  purpose  should  be  concerned  with 
what  behavioral  change  will  take  place  should  the  purpose  of  the  course  be 
attained.    They  also  feel  that  these  purposes  should  be  stated  as  immediatG 
outcomes  of  instruction*  and  not  outcomes  to  be  reached  in  the  distant  future. 
This  process  of  identifying  the  purpose  will  help  teachers  to  make  clear 
statements  of  what  they  are  trying  to  teach.    Some  teachers  in  the  past  have 
had  a  clear  understanding  of  what  was  to  be  taught  and  what  was  to  be  learned 
by  the  students  and  were  able  to  translate  this  notion  into  relevant  learning 
experiences  without  ever  having  put  them  down  on  paper.    However,  many  other 
teachers  have  not  carried  their  thinking  beyond  the  point  of  selecting  the 
content  to  be  presented.    They  have  not  considered  carefully  what  the  students 
are  to  do  with  the  information. 

Operational  ism  and  Behavioral  Objectives 

Operational  ism  is  a  concept  borrowed  from  the  hard  sciences.    It  is  con- 
cerned with  banishing  ambiguity  and  obscurity  from  the  language  of  science- 
By  applying  scientific  concepts  to  concrete  procedures  one  could  avoid 
inconsistent  and  contradictory  meanings.    In  the  context  of  objectives  it 


refers  to  the  process  of  defining  abstract  constructs  or  concepts  in  terms  of 
a  limited  number  of  Instances  drawn  from  the  three  domains  of  learning: 
cognitive,  affective^  and  psychomotor  (Tieman,  1977)*    Tuckman  (1972)  says 
an  operational  definition  is  a  definition  based  on  the  observable  character^ 
istics  of  that  which  is  being  defined.    In  the  field  of  behavioral  research, 
operational  definitions  are  formulated  so  that  statistical  methods  can  be 
applied.    These  methods  produce  reportable  evidence  and  hard  conclusions. 
The  behavioral  objectives  approach  requires  that  behavioral  objectives  be 
precisely  stated  in  behavioral  terms. 

Opposing  View  Points  Concerning  the  Definition 

Some  educators  feel  that  there  1s  difficulty  with  explicitly  defining 
behavioral  objectives.    MacDonald-Ross  (1973)  points  out  that  some  of  the 
problems  encountered  In  the  behavioral  objective  domain  are  extensions  of 
the  basic  problems  faced  by  operatlonalism.    He  states:    "What  exactly 
counts  as  an  operation?   What  happens  to  the  concepts  when  we  are  not  per- 
forming operations  or  If  we  have  not  yet  learnt  how  to  perform  them?" 
Hempel  (1958)  says  that  the  greatest  advances  in  scientific  systematisation 
have  not  been  accomplished  as  a  result  of  referring  explicitly  to  observable 
behaviors s  but  rather  by  means  of  laws  that  speak  of  various  hypothetical  or 
theoretical  attributes*    He  points  out  that  activities,  events s  and  attitudes 
which  are  not  ascertainable  by  direct  observation  have  an  important  and  valid 
place  in  the  educational  system.    For  Instance,  in  the  fine  arts  it  is 
extremely  difficult  to  have  an  observable  product  when  judgment,  feeling, 
and  creativity  play  such  a  major  role.    MacDonald-Ross  points  out  that  as  far 
as  art  subjects  are  concerned,  there  are  no  ultimate  goals  to  be  reached,  but 
rather  standards  of  judgment  and  tastes  to  be  developed.    He  also  says  that 


these  broad  goals  in  the  arts  do  represent  a  type  of  behavlorp  which  being 
internal  is  not  observable*    Eisner  (1967)  supports  the  positions  that  attitude 
values,  and  creative  experiences  are  important  educational  aims  which  cannot  be 
translated  into  behavioral  terms.    Burns  (1972)  feels  that  if  the  definition 
of  behavioral  objectives  is  concerned  only  with  specific  behaviors,  there  is 
no  room  for  expansion^  self  discovery^  original itys  and  whatever  you  might 
wish  to  call  that  which  Is  subsumed  under  the  general  term  "creativity," 

Research  Related  to  the  Development  of  a  Consistent  Operational  Definition 
A  series  of  studies  (Barron,  Gerlachi  and  Haygood,  1976^  and  Haygood, 
Gerlach  and  Wlgand^  1977)  deal  v/lth  analyzing  rater's  perception  of  the  com- 
ponents of  behavioral  objectives,  rated  both  In  isolation  and  within  complete 
statements  of  objectives*    These  studies  measured  the  degree  to  which  the 
various  components  contribute  to  the  raters'  perceptions  of  the  complete 
objective.    These  empirical  studies  have  currently  Investigated  the  development 
of  a  consistent  operational  definition  of  the  terms  behavioral  objective. 
The  results  indicate  clearly  that  no  single  component,  the  verb,  direct  object, 
condition,  or  standard,  should  be  singled  out  as  being  of  primary  importance 
in  determining  the  character  of  a  behavioral  objective*    Investigators  are 
moving  closer  towards  a  consistent  operational  definition  of  the  behavioral 
objective,  but  additional  research  is  needed  that  will  limit  the  many  dis- 
crepancies among  educators  concerning  the  definition* 

II  Form 

Major  Theorists  Views  on  Form 

Many  articles  and  books  have  appeared  in  the  professional  literature 
concerning  the  proper  form  of  behavioral  objectives  (Mager,  1962,  Bloom,  1964, 


b 

LindvalU  1963,  Popham  and  Baker,  1970,  and  Kibler,  Barker,  and  Miles,  1970). 
Tyler  (1934)  suggests  one  should  state  the  objectives  in  such  clear  and 
definite  terms  that  they  can  serve  as  guide  for  constructing  test  questions. 
Many  statements  of  objectives  are  so  vague  and  nebulous  that  they  prove  to 
be  glittering  generalities  which  are  of  little  value  as  guide  In  teaching  and 
of  no  value  in  making  examinations.    Mager's  (1962)  three  criteria  for  a  well 
stated  behavioral  objective  are  probably  the  best  known r    (1)  One  should  state 
the  objective  in  terms  of  v/hat  the  learner  will  be  able  to  do  after  the 
learning  experience.    This  is  done  by  selecting  verbs  which  describe  observable 
actions.    Such  words  as  Identify,  describe,  construct,  and  list  are  far  less 
ambiguous  than  verbs  such  as  to  know,  understand,  or  appreciate.    (2)  The 
second  characteristic  of  a  well  stated  objective  is  a  statement  of  the  condi- 
tions under  which  the  perforfnance  is  to  occur.    Conditions  should  be  stated 
clearly  enough  that  others  understand  your  intent  as  you  understand  it. 
(3)  The  third  characteristic  of  a  well  stated  objective  is  the  criterion,  the 
quality  or  level  of  performance  that  will  be  considered  acceptable- 

Some  educators  feel  that  Mager's  criteria  for  a  well-stated  behavioral 
objective  have  weaknesses.    Merrill  (1970)  reports  that  Mager's  criteria  for 
a  welNstated  objective  fail  to  distinguish  the  level  of  behavior*    He  state?: 
that  there  are  more  purposes  to  Instructional  objectives  than  transmission  of 
knowledge  and  increasing  proficiency*    He  also  points  out  that  there  are  two 
classes  of  conditions  under  which  behavior  is  to  occur.    The  first  is  concerned 
with  those  conditions  related  to  a  particular  subject  matter  and  unique  to  the 
testing  situation.    An  example  of  a  condition  stated  In  a  behavioral  objective 
for  a  math  class  would  be        using  only  a  calculator.,."  or        using  only 
the  protractor..,"    The  second  Is  concerned  with  the  psychological  conditions 
which  help  define  the  behavior  being  observed.    This  second  type  is  quite  often 
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overlooked  and  is  more  important  because  the  type  of  behavior  being  observed 
will  change  when  psychological  conditions  are  changed.    In  most  cases  the 
psychological  conditions  are  not  stated  in  the  objective,  but  have  an  impor- 
tant effect  upon  its  outcome.    For  example^  the  classroom  learning  environment 
is  typically  not  normal  the  day  before  Christmas  vacation.  MacDonald-Ross 
(1973)  feels  that  a  fourth  characteristic  should  also  be  considered  when  deter- 
mining what  constitutes  a  well  stated  objective:    an  objective  should  be 
relevant  to  the  general  educational  aims  of  a  course*    He  states ,  "No  rules 
are  given  for  achieving  this  criter1on--which  is  actually  the  most  difficult 
to  achieve,  yet  the  most  important  of  all"  (p.  12). 

Gagn#  and  Briggs  (1974)  agree  with  the  three  basic  criteria  set  forth 
by  Mager  and  later  writers  concerning  a  well-stated  behavioral  objective, 
Gagne  and  Briggs  also  state  that  the  choice  of  verb  in  an  objective  is  a  matter 
of  critical  importance.    They  feel  that  there  are  two  kinds  of  verbs  which 
must  be  incorporated  into  an  objective*    The  first  verb  denotes  action*  Verbs 
denoting  action  are  not  difficult  to  find.    Common  ones  are,  writes,  drav^fSs 
selects s  matches,  names,  groups ^  verifies*    There  are  many  others  as  well* 
The  following  examples  denotes  action:    Without  use  of  reference  materials ^ 
state  the  provisions  of  the  Fifth  Amendment,  in  writing.    While  it  may  be 
essential  for  completeness  of  communication ^  is  not  necessarily  the  most  impor- 
tant verb  in  the  definition  of  an  objective.    The  second  verb,  (i.e,,  the 
major  verb)  which  they  feel  is  probably  of  even  greater  importance  In  its 
implications,  denotes  learned  capability*    It  has  the  purpose  of  communicating 
the  kind  of  human  capability  one  expects  to  be  learned^  as  it  may  be  observed 
in  some  performance  exhibited  by  the  student*    The  following  verbs  describe 
performances  Implying  learned  capabilities:    discriminate,  classify,  demons 
stratGj  generate,  execute,  originate.  Identify,  and  state*    Several  examples 


that  use  verbs  which  describe  learned  capabilities  are:        •states  orally 
the  major  Issues  1n  the  Presidential  campaign  of  1968,"  and        identifies , 
by  naming,  the  root,  leaf,  and  stem  Of  representative  plants . Early  writers 
regarded  the  verb  as  the  primary  determiner  for  which  objectives  were  con» 
sidered  behavioraK    Many  writers  provided  us  with  lists  of  verbs.    Not  until 
Deno  and  Jenkins  (1968)  was  there  any  empirical  data  collected  regarding  the 
verb  and  behavioral ity.    Deno  and  Jenkins  selected  a  list  of  verbs  from  a 
well-known  experimental  curriculum-    They  had  a  group  of  elementary  and 
secondary  teachers  rate  the  verbs  on  a  five  point  scale  of  observability. 
The  following  results  were  reported  by  Deno  and  Jenkins     'The  results 
indicate  that  many  widely  used  and  recommended  behavioral  terms  refer  to 
behavior  which  is  not  regarded  by  teachers  to  be  as  clearly  'observable*  as 
some  have  suggested"  (p.  22),    They  concluded  that  verbs  used  in  behavioral 
objectives  are  selected  for  usage  rather  than  observability*    Gerlach  (1974) 
replicated  the  Deno  and  Jenkins  study,  by  rating  the  same  ninety-nine  verbs. 
The  results  obtained  by  the  study  were  basically  the  same  as  those  reported 
by  Deno  and  Jenkins, 

A  Closer  Look  at  the  Three  Essential  Characteristics  of  a  Behavioral  Objective 

A  verb  which  describes  overt  behavior  Is  the  main  factor  Involved  in 
stating  clear  descriptions  of  what  the  learner  must  do  to  perform  the  task. 
There  are  many  verbs  which  could  be  used  In  behavioral  objective  statements. 
The  following  list  Is  illustrative  of  widely  used  verbs:    identify,  name, 
describe*  construct,  state,  discriminate,  classify^  generate,  name^  order, 
check,  and  perform  (Sullivan,  1969,  Deno  and  Jenkins,  196S,  Gagne  and  Briggs^ 
1974,  Gerlach,  1974), 


The  choice  of  verb  in  an  objective  is  a  matter  of  critical  importance. 
The  primary  reason  is  the  avoidance  of  ambiguity.    The  statement  of  an  objec- 
tive should  comiunicate  reliably^  in  such  a  way  that  two  different  literate 
people  will  agree  on  the  specific  behavior  which  is  to  be  exhibited  by  the 
learner.    Words  such  as  "knowSp"  "understands 5"  ''appreciates,'*  do  not  communi- 
cate reliably.    The  action  should  be  expressed  in  the  objective  so  that  anyone 
who  reads  it  will  be  able  to  identify  the  same  performance. 

The  statement  of  conditions  which  specifies  the  condition  under  which  the 
behavior  is  to  occur  is  the  second  essential  characteristic  of  a  well-stated 
behavioral  objective*    The  conditions  specify  the  limitations  and  restrictions 
which  are  Imposed  on  a  learner  when^  following  instruction^  he  performs  the 
task  stated  in  the  objective.    Conditions  describe  the  materials,  events, 
Information^  and  the  objects  in  the  learner's  environment.    Examples  of  stimulu 
conditions  include  the  following: 

"When  presented  with  a  typed  list,,." 

"With  the  use  of  class  notes.,," 

"Without  the  use  of  classnotes  or  other  references..," 
Ambiguity  is  reduced  when  precise  limitations  and  restrictions  are  specified. 

The  third  essential  characteristic  of  a  well  stated  behavioral  objective 
Is  the  statement  of  criterion,  which  describes  h^w  well  the  learner  is  to  per-- 
form  the  task.    The  criterion  or  standard  provides  a  basis  for  evaluating  the 
prescribed  behavior.    For  example,  consider  the  objective  "Name  the  four  major 
food  crops  grown  in  Arizona."    The  standard  is  "correctly  name  all  four  major 
food  crops  grown  in  Arizona  and  only  those  four.    Thus  a  performance  standard 
is  a  specified  level  of  achievement  used  to  determine  whether  or  not  a  task 
has  been  mastered  satisfactorily.    Performance  standards  help  both  teachers 


and  students  know  where  any  given  student  Is  in  a  program,    Mager  (1962) 
states^  "When  the  minimum  acceptable  performance  for  each  objective  is  speci- 
fied we  have  a  performance  standard  to  use  In  assessing  students'  work. 

Mastery  generally  means  that  the  student  will  exhibit  the  performance 
100%  of  the  time  (minus  some  small  percent  for  "measurement  error").  However, 
frequently  it  is  appropriate  to  set  a  lower  standard,  such  as  three  out  of 
five  problems  solved  correctly,  or  four  out  of  six  defects  identified  (Bloom, 
1971  )•    Briggs  (1970)  states,  "Many  people  find  the  how  well  criterion  the  most 
awkv/ard  to  include  in  a  statement  of  objectives.    But,  for  objectives  requiring 
more  complex  evaluation,  it  may  be  easier  to  omit  this  third  criterion  from 
the  actual  statement  of  the  objective,  and  present  it  in  the  scoring  key,  and 
grade  conversion  guide,  showing  just  what  standard  of  student  performance  will 
be  considered  acceptable," 

Behavioral  objectives  do  not  state  in  quantitative  terms  what  criteria 
will  be  used  to  determine  whether  or  not  the  objective  has  been  satisfactorily 
met  (Gagne  and  Briggs,  1974),    The  objective  does  not  say  how  many  times  the 
student  is  to  "demonstrate  the  addition  of  whole  numbers,"  or  how  many  "errors" 
will  be  permitted.    They  do  not  state  what  will  be  needed  for  the  observer  to 
be  confident  that  the  objective  has  been  met,    Gagnl  and  Briggs  feel  that  there 
are  two  reasons  why  the  criteria  should  not  be  included  in  the  objective  state- 
ment.   First,  the  criteria  specified  in  an  objective  is  not  Hkely  to  be 
applied  in  the  same  manner  to  all  individuals.    Second,  the  question  of 
criteria  of  performance  Is  a  question  of  "how  to  measure,"  and  is  bound  up  with 
the  techniques  of  performance  assessment.    At  the  point  in  the  instructional 
planning  when  objectives  are  being  described,  it  is  confusing  to  become 
concerned  with  assessment  procedure.    The  concept  of  mastery  implied  by  the 
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objective  statement  Is  derived  from  an  important  theoretical  viewpoint*  The 
theory  underlying  Gagne's  (1970)  learning  hierarchies  accounts  for  the  func- 
tion of  mastery.    According  to  the  theory,  the  achievement  of  an  Intellectual 
skill  is  important  because  1t  supports  the  learning  of  more  complex  skills. 
Strictly  speaking  from  a  practical  views  Gagnl  and  Briggs  point  out  that  it 
is  not  possible  to  predict  in  precise  terms  how  mastery  should  be  measured. 
They  state,  "It  is  not  wise  to  adopt  some  arbitrary  standard  like  five  out  of 
six  correct  responses.    The  criterion  of  mastery  will  vary  with  what  is  being 
learned^  and  needs  to  be  determined  as  a  part  of  the  assessment  process"  (p,  89) , 

The  roles  of  the  three  basic  components  of  an  objective,  as  stated  by 
Mager  (1962)  have  been  researched  by  Barron  and  Gerlach  (1974)*    Their  results 
confirmed  the  linportance  of  the  verb  1n  objectives^  but  they  also  found  that 
the  choice  of  conditions  and  criteria  Influences  the  rating  of  a  complete 
objective,    Haygood  et  al,  (1977)  state  that  "no  singhi  component^  such  as  the 
verbs  should  be  singled  out  as  being  of  primary  Importance  in  determining  the 
characteristic  of  a  behavioral  objective." 

Some  writers  contend  that  form  should  be  considered  only  as  a  function  of 
an  objective.    There  is  little  point  In  requiring  a  teacher  to  write  an  objec- 
tive in  standard  form  without  taking  into  account  the  purpose  for  stating  the 
objective  (Harlen^  1972).    This  purpose  should  provide  the  basis  for  teachers 
to  make  decisions  in  their  everyday  work  In  guiding  learning  in  the  classroom. 
Of  course  there  will  be  a  variation  from  one  teacher  to  another  as  to  the  form 
of  expression*    Harlen  states,  "The  form  in  which  the  objectives  are  stated 
must  also  be  left  to  the  teachers ^  trying  to  specify  them  in  the  detail  advo- 
cated by  Mager  may  be  of  help  to  some  in  encouraging  clarity  of  thoughts  it  may 
be  unnecessary  for  others"  (p,  234), 


Instructional,  or  classroofii,  objectives  are  priniarlly  the  responsibn itj^ 
of  teachers  and  cannot  be  determined  by  anyone  else  (Harlens  1972).  Unfor- 
tunatelyp  teachers  are  frequently  untrained  in  the  use  or  formulation  of 
behayioral  objectives  in  which  case  outsiders  suggest  examples  or  provide 
guides  to  defining  objectives.    This  may  do  as  much  harm  as  good  If  teachers 
accept  others'  objectives  as  their  own,  or  if  they  go  through  a  superficial 
training  which  teaches  them  the  form  but  not  the  philosbphy  behind  the  concept 
of  objectlvis  (Marlens  1972). 

Many  teachers  are  now  being  given  the  opportunity  to  implement  a  developed 
objective  based  instructional  program-    Niedermeyer  and  Sullivan  (1977)  state 
that  teachers  do  have  the  option  of  accepting  or  rejecting  an  objective  based 
program  on  their  judgment  of  the  worth  of  its  objectives  and  resources  for 
their  pupils.    There  Is  no  explicit  requireinent  that  teachers  are  to  use  all 
the  materials  that  have  been  developed  for  them^  or  closely  follow  the  recom- 
mended instructional  procedures.    The  intention  of  the  teacher  and  the  prograni, 
however 5  is  to  produce  successful  pupil  performance  on  the  objectives.  To 
reach  this  goal  teachers  should  use  whatever  resources  and  creative  abilities 
they  have, 

III    Function  of  Behavioral  Objectives 

The  functions  of  behavioral  objectives  can  be  divided  into  four  categories: 
(1)  aid  in  design  of  developing  efficient  instructional  programs;  (2)  provide 
guidance  in  evaluation  of  instructional  progranis;  (3)  facilitate  learning  for 
students;  and  (4)  inform  teachers ^  administrators^  and  general  public  of  the 
purposes  of  the  Instructional  projram, 

Ajd  in  Design  of  Instruction.    Objectives  offer  a  systematic  means  of 
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planning  in  education.    When  designing  a  program  or  system,  one  needs  to  know 
what  a  successful  solution  will  look  like  as  well  as  what  criteria  it  must 
satisfy.    MacDonald-Ross  (1973)  implies  that  behavioral  objectives  can  provide 
the  only  possible  rational  basis  for  evaluating  the  success  of  the  learning 
experience.    The  course  becomes  successful  only  if  the  students  can  demonstrate 
satisfactorily  what  the  objectives  predict.    He  also  states  that  objectives 
indicate  how  the  process  of  teaching  should  be  conducted  as  well  as  help  to 
assist  in  the  selection  and  design  of  instructional  activities. 

A  systematic  procedure  for  developing  instruction  has  been  developed  by 
Gagne  (1974),    He  states  that  when  objectives  nrm  ^nnwn    nna  ie  ahio  +0  ^n*.'^*" 
What  kind  of  learned  capability  is  being  acquired,  and  one  can  also  determine 
what  conditions  will  be  needed  to  bring  about  the  learning  with  greatest 
efficiency.    Clearly,  then,  the  systematic  design  of  lessons  which  make  up 
courses  will  result  in  the  development  of  a  sizeable  collection  of  statements 
of  objectives.    This  collection  of  objectives  wm  be  constructed  by  using 
such  schemata  as  Bloom's  Taxonomy  (1956)  or  Gagnl's  learning  hierarchy  (1970). 
Higher  level  objectives  will  be  formulated  which  will  depend  on  the  acquisi- 
tion of  lower  level  objectives.    These  lower  level  objectives  will  be  stepping 
stones  or  prerequisite  skills  that  will  have  to  be  mastered  before  the  higher 
level  goal  or  objective  can  be  achieved.    Thus  the  specification  of  prerequisite 
skills  should  provide  a  corriplete  description  of  those  previously  learned  skills 
needed  by  the  learner  in  order  to  acquire  the  new  skill  most  readny.  The 
identification  of  performance  objectives  makes  possible  the  classification  of 
capabilities  Into  useful  categories.    Without  these  categories,  we  can  deal 
with  learning  principles  only  on  a  very  general  basis.    With  them,  it  becomes 
possible  to  infer  what  kinds  of  learned  capabilities  are  being  acquired  at  any 

no 


given  point  in  the  learning  process.    One  can  also  determine  under  what  condi- 
tions (internal  as  well  as  external)  the  learning  experience  takes  place. 
This  knowledge  may  increase  the  efficiency  of  one^s  instruction. 

Instruction  is  to  be  designed  with  reference  to  performance  objectives  and 
the  prerequisite  capabilities  they  imply.    The  first  necessary  component  in  the 
design  of  Instruction  is  to  classify  the  lesson  as  having  a  particular  type  of 
learning  objective*    Along  with  specifying  objectivess  two  other  components 
are  included  in  the  design  of  instruction.    One  is  developing  methods  *  designing 
materials!  producing  mediae  and  developing  learning  experiences  or  exercises; 
the  other  is  evaluating  the  success  of  the  learners  after  the  instructional 
process.    Mager  (1968)  describes  the  three  components  of  instruction  In  an 
easy  to  remember  format:    (1)  Where  am  I  golng?^  which  refers  to  how  to  achieve 
the  objectives;  (2)  How  will  I  get  there? ^  which  refers  to  how  to  achieve  the 
objectives;  and  (3)  How  will  I  know  when  I've  arrived? ^  which  is  the  evaluation 
process  of  determining  whether  or  not  the  student  has  satisfactorily  achieved 
the  objective*    These  three  questions  can  be  used  when  planning  instruction. 

One  does  not  have  to  proceed  in  any  given  order  when  developing  the  three 
components.    MacDonald-Ross  (1973)  feels  this  would  be  entirely  too  mechanical 
a  view  of  the  procedure  of  instructional  design.    Rather,  he  suggests  that 
the  designer  should  do  his  best  1n  developing  objectives ^  then  move  on  to 
considering  the  end-of-un1t  tests ^  and  then  select  and  develop  the  instructional 
materials.    This  procedure  would  be  carried  out  until  one  felt  that  each 
component  had  been  specified  as  clearly  as  possible.    At  this  point  one  would 
develop  a  first  draft;  quite  frequently  there  will  be  changes  in  some  of  the 
objectives.    But  of  course  one  would  be  doing  this  on  the  basis  of  some 
evidence  rather  than  on  the  basis  of  some  vaguely  conceived  or  haphazard 


scheme.    One  should  realize  that  although  objectives  are  intended  to  be  a 
basis  for  prescribing  course  structure  and  evaluation,  frequent  adjustments 
must  be  made  in  practice. 

There  are  varied  viewpoints  concerning  the  advantages  and  disadvantages 
of  using  behavioral  objectives  in  instructional  design*    Baker  (1974)  for 
example^  feels  that  the  use  of  behavioral  objectives  in  forming  a  basis  for 
restructuring  instructional  progranis  may  have  some  negative  consequences. 
Because  objectives  are  stated  in  operational  language p  they  appear  to  be 
more  teachable*    Objectives  may  look  achievable  1f  they  follow  the  formula: 
"Given.,. the  student  will  be  able  to..."p  but  such  is  not  always  the  case. 
Because  it  is  easy  to  transform  goals  into  the  accepted  behavioral  objectives 
formatj  examples  of  learning  may  be  casually  produced.    Baker  states  that 
many  supervisors  and  curriculum  specialists  feel  that  as  long  as  the  behavioral 
verb  has  been  supplied,  there  is  little  to  criticize.    She  also  states  that 
"most  behavioral  objectives  do  not  present  sufficient  cues  regarding  what  a 
teacher  should  alter  in  instruction  1n  order  to  facilitate  improved  learning*" 

Objectives  help  as  a  stimulus  to  clear  thinking  by  forcing  the  teachers 
to  think  in  specific  terms  rather  than  in  vague  ambiguities.  MacDonald-Ross 
(1973)  feels  that  this  is  a  prerequisite  for  any  system  of  design  or  planning 
and  that  such  thinking  yields  the  additional  benefit  of  revealing  value  judg- 
ments that  might  otherv/1se  remain  concealed.    Once  external i zed ,  such  thinking 
can  be  subjected  to  criticism  and  testing,  and  thus  instruction  can  be  improved. 
Since  objectives  can  provide  a  stimulus  for  clear  thinking,  they  can  help 
teachers  in  developing  Instructional  goals,  strategies,  purposes^  and  methods. 
Kiblerj  Cegala,  Parker,  and  Miles  (1974)  suggest  that  if  teachers  state  their 
instructional  intent  in  behavioral  objectives,  other  teachers  will  be  able  to 


understand  what  content  is  being  covered  within  their  classroom. 

Although  there  1s  not  complete  acceptance  among  educators  of  the  specific 
use  of  behavioral  objectives  in  designing  instruction,  it  can  be  agreed  that 
behavioral  objectives  can  provide  guidelines  for  teaching  and  can  lay  the 
foundation  for  a  systematic  approach  for  curriculum  planning. 

Guidance  in  Evaluation  of  Instruction,    Objectives  are  useful  In  the 
evaluation  process.    Gagne  (1970)  states  that  descriptions  of  objectives  are 
descriptions  of  what  must  be  observed  in  order  to  venfy  that  the  desired 
learning  has  taken  place.    Consequently,  statements  of  objectives  are  used 
for  assessing  student  learning*    Teachers  may  use  objectives  to  design  situa- 
tions within  which  student  performance  can  be  observed;  or  objectives  can  be 
used  as  a  basis  for  test  construction.    While  objectives  can  be  used  as  a 
basis  for  evaluating  students,  they  can  also  be  used  as  a  basis  for  evaluating 
instruction.    Since  objectives  are  directly  related  to  instructional  content, 
and  since  they  include  a  performance  standard ^  both  the  student  and  the 
teacher  can  know  the  quality  and  quantity  of  a  successful  performance.  If 
students  constantly  fail  to  meet  the  standard  specified  in  the  objective,  this 
can  help  the  teacher  to  evaluate  either  the  instructional  content  or  activities 
that  are  related  to  the  objectives  not  being  attained  by  the  students.  Revision 
of  the  instructional  content  and/or  activities  may  be  needed  at  this  time, 
Kibler  et  al.  (1974)  feel  that  there  are  primarny  three  functions  of  evaluation 
In  instruction:    (1)  student  achievement  of  instructional  objectives;  (2) 
evaluation  of  Instructional  materials;  and  (3)  evaluation  of  the  Instructor. 
Kibler  goes  on  to  say  that  while  both  norm-referenced  and  criterion-referenced 
testing  can  be  used  to  provide  information  concerning  the  three  functions, 
criter1on»r€ferenced  testing  is  best  suited  for  accomplishing  functions  one 
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and  two,  and  norm- referenced  testing  is  best  suited  for  function  three.    It  is 
extremeiy  important  that  teachers  be  able  to  determine  the  student's  level  of 
achievement  at  any  time  during  an  instructional  program.    This  can  be  done 
very  effectively  by  keeping  an  accurate  record  of  the  students*  progress  on 
each  performance  objective  throughout  the  instructional  program.    When  teachers 
have  this  type  of  Information,  they  will  know  how  the  student  1s  performing  at 
any  time  and  they  will  be  able  to  pinpoint  any  weaknesses  in  learners. 

Evaluation  can  occur  throughout  the  instructional  process,    Briggs  (1970) 
states  that  tests  over  competencies  of  an  objective  are  useful  for  determining 
whether  or  not  students  need  additional  remedial  work.    They  also  are  a  useful 
source  for  pinpointing  trouble  when  a  student  fails  the  test  of  a  specific 
behavioral  objective.    When  tests  are  given  for  specific  objectives,  then  can 
serve  as  a  guide  for  the  teacher  in  detennining  whether  the  student  is  ready 
to  go  on  to  the  next  objective.    Tests  for  units  of  instruction  can  reveal  the 
learner's  mastery  of  more  complex  objectives*    End-of-course  tests  can  Indi- 
cate the  students'  ability  to  solve  more  complex  problems  or  to  apply  their 
knowledge  to  a  wider  range  of  situations.    Gagne  (1974)  feels  the  pre-tests 
based  upon  criterion  objectives  can  also  help  to  Identify  students  who  have 
acquired  the  level  of  performance  before  Instruction  begins.    Such  objectives 
may  also  help  in  identifying  students  who  lack  the  pre-requi sites  to  satis-- 
factorily  meet  the  criteria  set  forth  in  an  objective* 

Teachers  have  few  rules  to  go  by  when  writing  test  items;  selection  of 
content  is  often  haphazard.    Consequently,  when  teachers  are  faced  with  a 
student  who  has  not  satisfactorily  met  the  objectives,  they  often  have  dif- 
ficulty in  selecting  content  for  practice  items.    Teachers  usually  guess,  after 
inspecting  the  test,  what  relevant  class  of  examples  they  may  use  that  will 


correspond  to  the  objective*    Baker  (1974)  feels  the  a?isv;er  to  pmvlfiing  data 
to  facilitate  improveniftnt  of  instructional  programs  lies  in  domain  re^ferenced 
testing.    This  t^^pe  of  testing  can  suppli^  both  the  data  needed  for  as  sessnient 
uf  Instructional  programs  and  information  suitable  for  feedback  to  teachers 
to  facilitate  planning.    The  use  of  domains  in  the  desn'gn  of  tests  tie  Ips 
reduce  the  production  of  trivial  objectives,    A  doinain  consists  of  a  subset 
of  knowledge^  skills,  understarding  or  attitudes  where  the  essential  elements 
of  the  contents  in  which  the  student  Is  expected  to  acquire^  fs  carefully 
described.    Baker  states^  •'dornalns  for  teaching  and  testing  represent  an 
attempt  to  find  a  reasonable  compromise  bety/een  vagueness  and  over-pr-€Cls1on" 
(p.  n).    Domain  require  the  teacher  to  focus  on  the  range  of  eliglbliG  content 
to  which  the  learner's  skill  is  to  apply.    Designation  of  conterit  rul«s 
represents  the  major  difference  between  domain'- referenced  testmg  and  objective'- 
based  evaluation.    Content  linilts  pro^^ide  a  set  of  rules  to  desc ri be  wihat 
content  is  appropriate  to  Include  or  to  sample  nn  the  test  on  instructional 
examples*    The  content  limits  describe  tJie  range  of  content  to  wMch  ithe 
learner  is  expected  to  respond. 

Referring  again  to  the  three  components  in  the  desigri  c^f  instruction 
(see  p.  15)  j  it  can  be  seen  that  instruction  is  cycrlcal.    That  is  ^  tN  three 
coniponents  are  in  constant  feedback  loops.    Not  only  doies  the  firishecl 
product  get  tested  and  revised^  but  even  the  objectives  thense l\^es  ^re  subject 
to  revision.    The  resijlt  of  such  cycling  is  that  the  objecti  vess  coiirse 
content^  and  tests  may  eventually  form  parts  of  an  Interlocking  systeifij  where 
changes  In  one  part  will  require  adjustments  in  the  other  tv/o  parts.  The 
advantage  here  is  that  the  system  can  continue  Imp  rowing  c?ver  €  period  of  time. 
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FaciJltitgs^  Student  Learning.    When  behavioral  objectives  are  given  to 
students  prior  to  the  instructional  content  which  is  to  be  presented »  they 
provide  guidance  to  the  student  1n  the  processing  of  information,  Deterline 
(1968)  says  that  if  students  are  told  precisely  what  the  objectives  are,  in 
the  form  of  minimum  performance  requirements,  and  if  they  are  given  sample 
test  questions,  performance  can  be  improved.    Behavioral  objectives  provide 
goals  which  are  definable  and  assist  1n  guiding  the  teacher  in  developing 
student  activities.    If  behavioral  objectives  are  used  to  tell  the  student 
exactly  how  he  is  going  to  be  tested,  the  threatening  aspect  of  the  test  will 
probably  be  reduced.    Testing  is  a  means  by  which  students  can  check  on  their 
progress  5  or  as  a  tool  which  the  teacher  uses  to  help  them  progress.  By 
providing  the  objectives  to  the  students  one  is  conirTiunl eating  to  the  students 
what  they  are  to  do,  to  achieve  satisfactorily.    Kapfer  (1970),  too,  advocates 
presenting  behavioral  objectives  to  students.    He  says  that  if  students  are 
given  objectives  they  will  be  able  to  make  intelligent  choices  concerning  how 
they  will  attain  them.    In  the  past,  students  have  not  had  this  opportunity, 
Kibler  et  aK  (1974)  state  that  it  seems  reasonable  that  students  who  are 
presented  with  behavioral  objectives  are  spared  the  frustration  and  time- 
consuming  effort  of  trying  to  guess  what  the  teacher  expects  of  them.    It  also 
seems  logical  that  students  will  learn  more  if  they  are  told  what  1s  expected 
of  them  and  how  they  will  be  expected  to  demonstrate  that  they  have  satis- 
factorily met  the  objective. 

There  are  several  more  reasons  for  providing  objectives  to  students 
(Duchastel  and  MerrilU  1974).    The  first  is  that  behavioral  objectives  may 
provide  direction  to  students'  learning.    Since  they  will  knov/  exactly  what  is 
OKpected  of  them,  they  will  be  able  to  discriminate  betv/een  relevant  and 


non-relevant  material.    The  second  is  that  objectives  may  provide  some  organi- 
zation or  general  structure  to  the  content  or  subject  matter.    Duchastel  and 
Merrm  also  point  out  that  objectives  may  serve  a  management  function  by 
enabling  the  students  to  better  organize  their  time  and  learning  experiences 
in  terms  of  the  goals  of  the  course.    This  might  help  eliminate  the  typical 
cramming  sessions  which  often  precede  tests.    Another  function  is  that  of 
providing  learners  feedback  1n  terms  of  the  criteria  set  forth  in  the  obaective 
enabling  students  to  deal  with  any  discrepancies  between  performance  and  goal. 
Final ly*  presenting  objectives  to  students  may  help  to  motivate  them.  Students 
who  know  that  they  have  satisfactorily  met  the  criteria  set  forth  In  the 
objective  will  probably  be  more  motivated  than  students  whose  only  reinforce- 
ment comes  from  a  grade  at  the  end  of  a  course.    Duchastel  and  Merrill  also 
point  out  that  presenting  objectives  to  students  v/ill  have  no  results  If  the 
students  pay  no  attention  to  them  in  the  learning  situation.  Therefore^ 
teachers  must  make  an  effort  to  thoroughly  explain  the  meaning  of  objectives 
to  students  so  that  they  v/ill  actually  use  them  while  learning.    A  discussion 
on  the  form  and  function  of  behavioral  objectives  would  be  helpful.  However^ 
teachers  must  be  careful  not  to  give  long  and  extensive  lists  of  objectives  to 
students.    This  may  overwhelm  and  confuse  them.    Such  a  list  v^ould  defeat  its 
own  purpose* 

Gagnl  et  al.  (1974)  agree  that  the  advantage  of  providing  objectives  to 
students  Is  that  it  informs  the  learners  of  their  goal.    Qagne  disagrees  with 
those  who  contend  that  when  one  communicates  an  objective  to  students 5  they  may 
be  inhibited  from  trying  to  meet  still  other  objectives  which  they  may  forfnu- 
late  themselves. 


Informs  the  Teacher,  Administrator,  and  the  Lay^itizen  of  Purposes  of 
Instructional  Prociram.    Accountability  In  education  has  gained  acceptance  from 
both  the  public  and  the  federal  government.    With  the  grov/1ng  involvement  of 
parent  groups  in  making  decisions  about  local  educational  systems,  it  is 
clear  that  some  form  of  accountability  is  needed.    The  public  should  be  aware 
of  the  exact  nature  of  what  learning  and  schooling  are  all  about.    This  type 
of  accountability  is  becoming  a  more  frequently  discussed  issue  in  education. 
Taxpayers,  parents,  funding  agencies,  and  legislators  are  all  extemely 
interested  in  having  some  type  of  proof  that  education  in  fact  Is  taking  place 
in  our  schools.    Are  schools  really  doing  what  they  say  they  are  doing?  Why 
should  any  educator  try  to  cover  up  what  is  being  taught?    How  and  what  should 
"Johnny"  know  and  do  by  the  end  of  the  school  year?   The  answer  to  these 
questions  is  quite  simple  and  straightforward:    Specify  the  objectives,  which 
in  turn  will  inform  these  people  about  what  we  are  doing  and  how  we  can  prove 
it.    To  achieve  the  balance  between  spending  and  student  learning  that  accounta-- 
bility  demands,  the  teacher  and  school  system  must  show  evidence  that  students 
have  learned  as  a  result  of  their  instruction.    Educational  accountability  can 
be  demonstrated  successfully  only  when  educational  goals  and  objectives  are 
precisely  identified  and  stated.    Kibler  et  al .  (1974)  say  that  the  use  of 
instructional  bojectives  will  allow  teachers  tc  convey  their  goals  to  their 
supervisors  and  school  boards.    Burns  (1972)  suggests  that  specifying  what  is 
to  be  learned  Is  obviously  the  function  of  objectives. 

In  order  to  defend  budgets  or  requests  for  funds,  administrators  and 
teachers  can  provide  the  content  of  courses  1n  objective  form  to  the  school 
board  and  thus  demonstrate  the  need  for  expenditures  to  the  board  in  more 
concrete  terms.    This  process  is  much  better  than  trying  to  provide  verbal  or 
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verbal-pictorial  representations  of  learning  situations  as  they  really  exist, 
because  board  members  are  often  too  far  removed  from  the  classroom.  Thus, 
instructional  objectives  may  provide  a  basis  for  logical  3  concrete  reasons  for 
spending  money.    Scott  (1974)  states  that  objectives  can  also  be  used  to 
explain  to  parents  or  to  the  community  the  philosophy  on  which  a  given  coursQ 
of  instruction  1s  based.    Parents  are  often  neglected  in  the  educational  pro- 
cess.   Howevers  parents  are  becoming  Increasingly  concerned  about  the  quality 
of  education  In  the  schools  and  are  becoming  more  involved  1n  the  educational 
process*    A  list  of  objectives  could  be  sent  home  to  the  parents  telling  them 
which  objectives  their  child  attained.    Parents  could  then  evaluate  the  pro- 
gress of  their  child  at  intervals  during  the  year  and  check  to  make  sui^e  that 
their  child  is  keeping  up.    This  would  help  inform  parents  about  the  content 
being  taught  as  well  as  the  childs*  growth  in  the  program.    These  objectives 
could  also  inform  parents  of  the  child *s  v^eaknesses  and  strengths.    Such  a 
procedure  would  be  quite  an  improvement  over  the  report  care  procedure 
commonly  used. 

There  appear  to  be  a  least  two  advantages  to  the  use  of  objectives  in 
most  teaching  situations.    Firsts  objectives  prompt  teachers  to  determine  the 
most  significant  aspects  of  the  subject  matter  to  be  learned.    The  second  is 
that  objectives  aid  in  establishing  criteria  for  the  measurement  of  classroom 
achievement.    Since  instructional  objectives  require  teachers  to  specify 
criteria  for  acceptable  behaviors  and  to  determine  in  advance  how  satisfactory 
performance  will  be  measured,  teachers  can  achieve  an  increased  sense  of 
security.    They  feel  more  secure  in  their  position  and  more  satisfied  with 
their  professional  contribution  when  they  are  confident  In  teaching  the  subject 
matter^  confident  of  the  subject  matter's  importance,  and  confident  that  the 
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measurenient  techniques  will  measure  whether  or  not  the  objectives  have  been 
met  satisfactorily. 

Instructional  objectives  are  important  at  two  levels  of  administration. 
The  administrator  who  is  in  charge  of  curricula  relies  on  objectives  to  insure 
that  content  and  subject  matter  are  covered  adequately  and  that  subject  matter 
between  courses  does  not  overlap  or  become  redundant*    Instructional  objec- 
tives also  promote  a  thread  of  continuity  among  related  courses*  Instructional 
objectives  developed  by  teachers  give  the  supervising  administrator  insight 
into  the  teachers*  philosophy  and  course  goals.    Teachers  can  collect  data  to 
determine  the  effectiveness  of  their  instructional  program  and  if  students 
are  continually  failing  to  meet  the  standards  set  forth  in  the  objectives ,  it 
may  be  a  result  of  poor  instruction.    This  in  turn  will  enable  administrators 
to  more  effectively  evaluate  teachers. 

IV   The  Cajes  for  and  Against  Behavioral  Objectives 

The  Case  for  Behavioral  Objectives.    Since  the  time  Mager*s  (1962)  classic 
book  on  Prggarin^  Instructional  Objectives  provided  a  major  stiniulus  to  the 
use  of  behavioral  objectives  in  the  field  of  education^  two  distinct  schools 
of  thought  emerged,  the  first  arguing  the  case  for  the  use  of  behavioral 
objectives!  and  the  other  against  the  use.    In  arguing  the  case  for  the  use 
of  behavioral  objectives  in  education^  a  large  number  of  claims  have  been  made. 
Proponents  of  the  use  of  behavioral  objectives  maintain  that  behavioral  objec- 
tives clearly  indicate  to  students  what  is  required  of  them,  and  as  a  result, 
student  performance  improves  (Gagne,  igJO,  Mager,  1968,  Popham  et  al.,  1969^ 
and  Tyler^  1964).    Objectives  can  also  provide  communication  between  the 
teacher  and  the  student*    Students  become  aware  of  where  they  are  going  and 
what  is  expected  of  them  when  objectives  are  given  to  them.    Objectives  work 


as  an  organiEer.    A  considerable  number  of  studies  collected  ernplrical  data 
which  indicate  that  giving  objectives  to  students  prior  to  instruction  will 
enhance  student  learning,    Dalis  (1970)  demonstrated  that  by  using  precise 
instructional  objectives  in  advance  of  instruction  enhanced  learning  of  high 
school  students  in  a  health  education  class.    The  study  implies ,  however,  that 
objectives  must  be  stated  in  precise  terms ^  otherwise  their  value  to  the 
learning  situation  is  doubtful.    Doty  (1968)  investigated  the  effect  of  pre- 
senting objectives  to  students  in  a  reading  class.    The  results  showed  that 
the  students  who  had  prior  knowledge  of  the  objactives  scored  significantly 
higher  on  a  posttest  than  did  students  who  did  not  have  prior  knowledge. 
Lawrence  (1970)  studied  two  groups  of  students  In  a  nursing  care  course^  while 
Engel  (1968)  studied  two  groups  of  students  in  a  mathematics  course.  Both 
studies  reported  that  the  group  who  received  the  behavioral  objectives  prior 
to  instruction  perfomied  significantly  better  on  a  posttest.    Blaney  and 
McKie  (1969)  divided  siKty  volunteers  Into  three  groups ,  a  behavioral  objec- 
tives groups  a  general  introduction  groups  and  a  pretest  group.    The  results 
showed  that  the  behavioral  objectives  group  did  significantly  better  than  the 
introduction  group  on  a  posttest.    The  results  also  showed  no  significant 
difference  between  the  pretest  group  and  the  behavioral  objectives  group  on  a 
posttest.    Students  in  a  college  economics  class  were  divided  into  two  groups. 
One  group  received  the  behavioral  objectives  and  the  other  did  not,  Tieman 
(1968)  reports  that  by  using  retention  scores  as  criterion^  the  behavioral 
objectives  group  scored  significantly  better  than  the  non^objecti ves  group. 

There  are  also  several  studies  that  have  shown  no  significant  differences 
between  groups  of  students  who  have  received  behavioral  objectives  and  groups 
who  have  not  (Boardmans  1970,  Smith,  1967,  and  Weinberg,  1970).  Therefore, 


the  general izabi 1 ity  of  providing  objectives  to  students  prior  to  instruction 
is  not  easily  detemil ned*    The  evidence  reported  here  demonstrates  the  corn'- 
plexity  of  the  Issue.    Howevers  it  has  been  shown  that  objectives  sometimes 
help  and  are  almost  never  harmful.    Therefore,  if  the  provisions  of  objectives 
are  relatively  inexpensive,  one  might  as  well  make  them  available  to  students 
(Duchastel  and  MerriVU  1973). 

Another  reason  for  using  behavioral  objectives  is  that  they  serve  as 
operational  aids,  basically  because  they  are  designed  in  terms  of  action 
(MacUonald'Ross ,  1973).    That  is,  they  act  as  a  mediuni  of  communication  or  a 
mechanism  for  informing  people.    Curriculum  design  is  developed  by  the  team 
approach  quite  often  1n  our  schools*    By  using  well  specified  guidelines,  in 
the  form  of  behavioral  objectives,  each  team  member  will  know  exactly  what 
is  being  asked  of  him,    Thus^  the  division  of  labor  can  become  a  much  easier 
task. 

A  third  claim  for  using  behavioral  objectives  is  that  by  specifying  the 
exact  behaviors  one  wants  the  students  to  exhibit,  the  teacher  1s  better  able 
to  select  appropriate  learning  activities  or  to  design  and  suggest  alternative 
Instruction  strategies  appropriate  to  the  individual  learner.    By  constructing 
objectives  that  meet  Mager's  criteria  for  a  well  stated  behavioral  objectivei 
the  teachers  will  be  guided  in  their  choice  of  selecting  Instructional 
activities  that  will  be  specific,  precise^  and  relevant  to  the  desired  outcoiiie. 
Because  of  the  systematic  approach  to  instruction  that  behavioral  objectives 
afford,  the  teacher  can  also  pinpoints  at  any  time  during  instruction^  those 
students  who  may  be  experiencing  difficulty  in  achieving  the  objective.    Thus ^ 
the  teacher  will  be  able  to  design  and  suggest  alternative  instruction 
strategies  appropriate  to  the  individual  learner*    In  this  sense,  the  use  of 


behavioral  objectives  serves  as  an  operational  aid  for  the  teacher  in  providing 
individual  treatment  for  students.    Because  the  outcomes  of  objectives  can  be 
replicated  J  treatment  can  be  Individualized,    This  means  that  students  with 
different  entry  characteristics  are  recognized  and  remedial  work  can  be 
provided  forthosewho  may  need  it.    Tests  developed  on  the  basis  of  objectiveSj 
provide  the  teacher  with  diagnostic  capabilities.    MacDonald-Ross  (1973)  states 
that  individualization  may  also  rriean  that  students  can  choose  their  own  way  to 
reach  the  objectives.    For  e^cample,  they  might  form  contracts^  which  are 
written  agreement  between  the  teacher  and  student,  to  teach  the  goal  or 
objectives  specified  by  the  teacher. 

Clear  and  well  sequenced  objectives  are  necessary  for  Individualization 
of  Instruction.    Through  testing^  the  teacher  identifies  where  the  child  is 
academically  at  different  times  in  the  instructional  program.    Piper  (1977) 
points  out  that  frequent  re-evaluation  is  needed  In  order  to  continue  moving 
the  child  along ^ in  the  instructional  sequence.    Careful  and  frequent 
recording  of  student  progress  is  needed  to  facilitate  the  quality  and  accuracy 
of  individualized  Instruction^  as  well  as  allowing  for  the  evaluation  and 
continued  Improvement  of  teaching  techniques. 

A  fourth  clairn  for  using  behavioral  objectives  is  that  they  play  a  major 
part  in  the  Objectives  Based  Instructional  Programs  that  are  being  implemented 
in  our  schools.    Classroom  verified  objective  based  programs  have  the  potential 
for  enabling  teachers  to  provide  students  with  the  sufficient  amounts  of 
practice,  feedback ,  and  self  correction  needed  to  acquire  competency  on 
objectives  is  still  very  new  to  many  teachers.    Therefore ^  objective-based 
programs  are  assisting  teachers  1n  successfully  promoting  pupil  attainment  of 
the  objectives  by  providing  guidelines  for  effective  teaching  procedure. 
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However,  the  time  that  the  teachers  use  to  develop  the  instructional  materials 
and  procedures  so  that  there  is  a  high  level  of  pupil  achieveinent  on  the 
objictives  requires  extensive  amounts  of  time,  money,  and  expertise.  Sullivan 
and  Niedermeyer  (1977)  point  out  that  if  teachers  want  students  to  achieve 
mastery  of  objectives,  teachers  must  also  be  provided  with  instructional 
materials  and  procedures  developed  especially  for  the  objectives  of  the 
lesson.    They  go  on  to  say  that  merely  providing  behavioral  objectives  to  the 
teachers  will  have  little  effect  on  the  learning  situation.  Teachers 
should  not  be  accountable  for  high  levels  of  pupil  perfomiance  on  objectives 
without  the  proper  development  of  Instructional  materials  and  procedures. 
Sullivan  and  Niedermeyer  conclude  that  with  the  Increasing  rate  of  objectives 
based  programs  in  our  schools,  there  is  need  for  empirical  evidence  to  support 
the  concept  of  Objective  Based  Instructional  Programs. 

A  fifth  claim  for  using  behavioral  objectives  is  that  they  provide  useful 
ifnormation  for  evaluating  curriculum  planning.    Objectives  guide  the  teacher 
and  student  in  the  teaching-learning  process  and  they  provide  a  measure  against 
which  progress  can  be  judged.    If  teachers  are  to  improve  their  teaching, 
they  must  have  information  that  determines  the  success  of  different  teaching 
methods  and  strategies  that  are  used  in  instruction.    Well-stated,  clear 
objectives  help  provide  this  information.    The  information  concerning  curriculum 
planning  evaluation  is  analyzed  by  using  either  norm-referenced  or  criterion- 
referenced  testing  procedures.    Norm-referenced  testing  compares  an  individual 's 
performance  with  that  of  a  normative  group.    The  standard  in  this  type  of 
testing  is  comparing  a  particular  student's  score  with  how  other  individuals 
performed  on  the  test.    On  the  other  hand,  criterion-referenced  evaluation 
procedures  are  designed  to  determine  whether  a  student  has  achieved  mastery 
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of  a  behavior  as  specified  in  an  instructional  objecti ve(s) .    In  criterion 
referenced  testing  the  interpretation  of  a  student *s  score  performance  is  in 
no  way  dependent  upon  the  performance  of  other  students.  Criterion-referenced 
procedures  assume  that  if  instructional  objectives  are  importants  teachers 
should  be  concerned  with  whether  students  have  achieved  thertis  not  with  how  much 
they  achieved  relative  to  their  peers.    Objectives  help  to  provide  information 
on  student  achievernent  throughout  the  unit  of  instructlona  as  well  as  the  end 
of  the  unit.    Kibler  et  aK  (1974)  point  out  that  criterion  referenced  testing 
is  used  for  at  least  four  different  types  of  testing  purposes:    (1)  for 
pre-assessment  purposes,  {2)  for  formative  testing--to  check  on  the  progress  of 
students  so  that  assistance  may  be  provided  when  necessary,  (3)  to  determine 
whether  components  of  instructional  model  need  modification ^  and  (4)  to 
determine  whether  students  have  achieved  the  criterion  levels  of  objectives  at 
end  of  Instructional  unit.    Teachers  can  determine  whether  students  are  ready 
to  go  on  to  higher  order  objectives  or  more  complex  objectives  by  evaluating 
lower  level  ones*    Accurate  assessinent  of  what  each  student  can  and  cannot  do 
is  critical  for  good  teaching.    It  is  a  waste  of  timej  as  well  as  frustrating , 
to  tell  students  what  they  already  know  about  the  content  being  presented  or 
to  present  information  that  is  "over  their  heads.'*    Teachers  have  difficulty 
determining  where  their  students  are  in  the  Instructional  process.  With 
objectives  clearly  in  mind,  much  of  the  guessv/ork  is  eliminated.    When  students 
see  that  they  have  achieved  a  satisfactory  performance  on  lower  level  objectives, 
they  are  encouraged  to  further  effort.    Clearly  stated  objectives  motivate 
both  teachers  and  students. 

Behavioral  objectives  may  be  informative  in  regard  to  curriculum  planning. 
Bruton  (1974)  has  done  research  on  whether  or  not  behavioral  objectives  would 


provide  information  for  curriculuiti  planning.    The  researcher  evaluated  a  v^idely 
used  set  of  oral  language  objectives  from  the  Distar  Lannuage  I  Program 
(Englemann,  Osborne^  and  Englemann,  1969)  for  use  with  first  grade  children. 
The  results  showed  that  fifty-four  of  the  eighty-five  objectives  were  shown 
to  be  useful  in  relation  to  instruction.    The  results  also  indicate  that 
thirty-one  of  the  objectives  were  not  applicable  for  classroom  use,    ThuSs  the 
research  cited  shows  that  objectives  can  be  used  for  guiding  student  learning. 

The  Case  Against  Use  of  Behavioral  Objectives.    Not  all  educators ^  by  any 
meanSi  favor  the  use  of  behavioral  objectives.    One  concern  is  that  of  the 
origin  of  object1ves"=how  are  they  derived?    MacDonald^Ross  (1973)  states  that 
there  1s  no  consistent  view  among  educators  as  to  the  origin  of  cbjectives. 
He  feels  that  two  schools  of  thought  have  emerged  concerning  methods  for 
deriving  behavioral  objectives.    One  group  attempts  to  provide  explicit  rules 
for  converting  observable  human  action  into  behavioral  objectives.  MacDonald- 
Ross  refers  to  this  group  as  the  "hardliners,      They  do  not  agree  with  the 
distinction  between  knowledge  and  skills  and  between  education  and  training. 
The  hardliners  believe  that  one  can  observe  a  "master  performer"  at  a  task 
and  be  able  to  prescribe  educational  objectives.    MacDonald-Ross  feels  that 
the  task  analysis  procedure  might  be  effective  for  skills  but  inadequate  for 
general  education.    For  example ^  trying  to  specify  the  objectives  for  a  course 
in  engineering  by  observing  master  performers  would  be  quite  fruitless  unless 
you  were  also  willing  to  take  into  account  the  network  of  knowledge  and 
understanding  underlying  their  actions.    MacDonald-Ross  states  "the  hardline 
case  thus  seems  to  fail.    It  Is  not  sufficient  to  use  observations  of  action 
(whether  of  action  at  work,  or  during  examinations)  for  a  prescription  of 
educational  objectives,  if  one  takes  the  meaning  of  the  word  'education*  at  all 
seriously. 


The  other  group  fully  accepts  that  educational  objectives  need  to  be 
derived  in  a  way  which  does  justice  to  the  difference  betwien  education  and 
training.    This  group  is  constantly  trying  to  justify  the  use  of  behavioral 
objectives  1n  instruction.    MacDonald-Ross  (1973)  refers  to  this  group  as  the 
"softliners."    Popham  and  Baker  (1970)  state  that  objectives  are  derived  from 
three  sources^    (1)  the  learner^  (2)  the  stociety,  and  (3)  subject  matter. 
They  propose  that  philosophy  of  education  and  psychology  of  learning  can  help 
one  formulate  general  objectives  which  can  then  be  developed  Into  precise 
instructional  objectives.    MacDonalH-Ross  feels  that  Popham  and  Baker's 
philosophy  is  frankly  "hilarious"  since  it  1s  far  f^'^om  being  an  operational 
procedure  for  deriving  behavioral  objectives.    All  the  critical  decisions  seem 
to  be  left  to  intuition  and  common  sense  of  the  teacher.    This  Joes  not  reflect 
the  concept  of  a  systematic  approach. 

Many  research  studies  in  the  area  of  behavioral  objectives  suffer  frorn 
the  lack  of  a  precise  operational  definition  of  what  a  behavioral  objective  is. 
Duchastel  and  Merrill  (1973)  and  MacDonald-Ross  (1973)  state  that  there  is  a 
lack  of  specificity  in  relation  to  determining  whether  objectives  are  behavioral 
or  non-behavioraK    Several  studies  Indicated  that  some  educators  use  Mager-s 
three  criteria  for  a  well  stated  behavioral  objective  when  defining  the 
objectives,  others  did  not.    Some  studies  gave  examples  of  well -stated 
objectives,  while  many  others  did  not.    This  lack  of  operational  definitions 
of  the  Dbjective  variables  makes  it  difficult  to  compare  studies. 

The  level  of  specificity  needed  in  constructing  behavioral  objectives  is 
another  concern  in  the  use  of  behavioral  objectives.    MacDonald^Ross  (1973) 
states  that  there  is  an  absence  of  rules  for  deciding  what  level  of  specificity 
objectives  should  be  developed.    General  objectives  have  a  tendency  to  become 


vague  and  ambiguous.    In  tidying  to  eliminate  ambiguity^  one  runs  the  risk  of 
writing  an  Impossibly  long  list  of  objectives,    Wight  (1973)  states  that  when 
teachers  begin  writing  objectives  for  what  they  hope  to  see  as  a  result  of 
instruction 5  they  often  write  a  behavioral  objective  for  each  Intended  outcome. 
This  can  result  in  a  list  so  long  that  is  more  an  obstacle  course  than  an  aid 
to  learning  for  the  student*    Wight  also  states  that  meeting  the  specification 
for  a  properly  written  objective  often  Inhibits  productive  thinking.  Teachers 
get  so  caught  up  in  the  mechanics  of  writing  behavioral  objectives  that  they 
lose  sight  of  their  priorities ^  namely ^  what  they  are  really  trying  to  teach. 
MacDonald-Ross  (1973)  concludes  that  "every  time  behavioral  objectives  have 
been  constructed  on  a  large  scale  this  problem  of  specificity  has  proved  quite 
fearsomely  difficult."    He  feels  that  no  satisfactory  rules  have  emerged  and 
perhaps  the  problem  of  specificity  is  insolvable  1n  principle.    Woodruff  and 
Kapfer  (1972)  state  thfit  the  level  of  specificity  in  constructing  objectives 
has  seemed  to  transform  education  Into  a  mechanistic  program  devoid  of  real 
human  value.    They  point  out  that  one  of  ^.he  most  coPDeVliuj  critics  of 
behavioral  objectives^  Charles  Sllvermon  (1970)^  states  "Indeed,  the  approach 
to  insturctional  technology  that  most  researchers  are  following  (based  on 
precise^  measurable^  behavioral  terms)  is  likely  ....  compound  what  is  wrong  with 
American  education-^its  failure  to  develop  sensitive  autonomous,  thinking ^ 
humane  individuals''  (p.  196). 

Triviality  is  another  criticisni  of  behavioral  objectives.    In  trying  to 
satisfy  the  requirernents  of  a  properly  written  behavioral  objective^  teachers 
quite  often  find  themselves  listing  Inconsequential  student  behaviors  and  over- 
looking Important  ones.    Meaningful  objectives  are  more  difficult  to  state  in 
prescribed  form.    Pascal  (1975)  states  that  many  educational  goals  which  can  be 


expressed  in  behavioral  objective  form  are  Irivi ^  vsfith  the  result  that  impor- 
tant outcomes  of  education  are  under-eFTipbasized.    Popham  (1968)  states  that  by 
going  through  the  process  of  constructing  ejcplicitly  stated  objecti\/es  ?  educators 
can  identify  and  v/eed  out  the  trivial  goals  ;  theii  they  can  begfn  paying  atten- 
tion to  more  meaningful  educational  objectives,    MacDonald-Ross  (1973)  states 
that  while  Pophani's  response  that  trivial  objectives  can  be  v^eeded  out  once 
revealed  may  be  true^  it  still  avoids  the  "fiuge"  probleiiis  of  origins  and 
operational  Ism.    He  feels  that  the  problem  of  triviality  is  stfll  a  probleni. 

One  of  the  most  fundamental  problems  with  behavi  oral  objectives  15  that 
the  objective'  itself  is  confused  wth  the  indicator  (means  of  daterrnin  ing 
whether  the  objective  is  achieved  (Wight^  1973).    A  behavioral  objecti  ve  Is  a 
statement  of  a  measurement  to  be  taken^  under  specified  conditions  v/it&i 
criteria  for  evaluation  to  act  as  evidence  that  the  desired  beliavior  h-^s  been 
achieved.    Many  educators  suggest  that  one  should  write  a  general  stat^ement 
or  goal  first.    Then  behavioral  objectives  cin  be  vir1  tten  v/hf  cli  rel ate  to  the 
general  statements.    But  the  general  goal  is  often  lost  because  of  the  focus  on 
the  behavioral  objective;  too  much  attention  Is  dev'Ot^d  to  ttie  present  per- 
formance specified  in  the  indicator  as  opposed  to  the  future  capabilities  called 
for  in  the  general  goal.    Special  effort  should  bi  matie  to  relate  the  indicator 
to  the  goal.    If  this  does  not  happen  the  student  may  ffnd  little  rneaning  in 
the  specification  of  performance. 

Behavioral  objectives  frequently  do  not  take  1  nt^  account  that  th^re  rnay 
be  niany  ways  of  assessf  ng  whether  an  objective  has  been  ac^iev&d.    Thi^  is 
particularly  true  when  dealing  with  higher  order  cognitf  ve  objectives  «r 
objectives  In  the  affective  donialn.    An  additional  Critfclsrn  of  behaviciral 
objectives  is  that  they  do  not  specify  the  masurement  to  be  used.    There  are 
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frequently  many  opportunities  for  assessing  the  behavior  demonstrated  within 
a  learning  activit)^  and  some  of  these  may  be  better  than  those  specified  in 
the  behavioral  objective.    It  may  be  argued  that  behavioral  objectives  do  not 
prevent  educators  from  seeking  other  evidence  to  verify  that  the  behavior  has 
been  achieved  satisfactorily.    But  the  point  is  that  a  well-stated  behavioral 
objective  has  a  measurement  component  that  is  explicitly  stated ,  which  causes 
the  teacher  not  to  look  further  for  additional  evidence  that  the  objectlva  has 
been  achieved. 

The  question  arises  as  to  which  behaviors  or  products  of  behavior  can  he 
accepted  as  valid  for  the  purpose  of  objectives?   Harlen  (  1972)  states  ''b#tevior 
by  definition^  is  observable,  but  two  observers  would  not  always  agree  on  the 
same  interpretation  of  a  particular  item  of  behavior*'  (p,  226),    Some  behaviors 
may  be  observed  only  under  cetain  special  condition,    Are  these  described 
behaviors  to  be  restricted  to  those  objectives  that  can  be  observed  and 
objectively  judged  only  under  certain  condition?    Harlen  points  out  that 
observable  behavioral  changes  are  only  sampled,  and  that  Indications  that  a 
change  in  behavior  has  taken  place  can  never  be  a  certainty.    He  feels  that 
there  is  a  blurred  line  between  what  is  accepted  as  observable  and  v/hat  is 
not.    Taba  (1962)  feels  that  deciding  v/hether  or  not  the  criteria  of  the 
objective  has  been  reached  is  less  Important.    Harlen  feels  that  the  standard 
of  acceptable  performance  In  a  behavior  objective  should  not  be  rigidly 
applied  and  that  some  "slack"  should  be  allowed^  otherwise  there  is  a  potential 
drawback  in  specifying  behavioral  objectives  with  explicit  criteria. 

Some  educators  are  expressing  a  distaste  for  the  whole  process  of 
defining  objectives.    Sheehan  (1974)  states  that  advocates  of  behavioral 
objectives  have  become  overly  zealous  and  preach  their  message  with  a 
vengeance.    Eisner  (1967)  states  that  many  educators  feel  that  the  specifi- 
cation of  objectives  encourages  students  to  seek  the  line  of  least  resistance 


and  thus  lower  their  om  educational  goals.    Other  educators  feel  that  there 
are  important  educational  aims  which  cannot  be  translated  into  behavioral 
terms—attitudes,  values  and  the  creative  experiences.    Still  others  feel  that 
the  heavy  emphasis  on  behavioral  objectives  irnplies  training  rather  than 
education.    Some  educators  believe  that  the  behavioral  objective  movement  will 
sterilize  education.    Sheehan  (1974)  states  "The  attempt  to  package^  to 
circumscribe^  and  to  modularize  materials  for  the  consumer  is  seen  as  counter- 
productive.   It  is  in  direct  conflict  with  the  more  important  goals  of 
teaching  students  to  identify  their  own  educational  requirements^  to  decide 
what  they  would  like  to  learn  and  what  is  important  to  them  within  the  Urnits 
of  vvhat  Is  available  and  their  own  abnities."    He  goes  on  to  say  that  the 
student  must  be  able  to  learn  on  his  own  with  the  help  of  the  following  tools: 
books,  films,  television,  journals^  colleagues,  his  own  observation^  and  the 
teachers.    He  must  derive  from  them  what  is  Important  without  the  aid  of 
superiniposed  Instructional  objectives,    Sheehan  feels  that  behavioral  objectives 
advocates  have  pushed  their  views  too  zealously  and  tend  to  see  obiectives  as 
ends  in  themselves  rather  than  as  being  only  a  small  part  in  the  instructional 
process, 

A  final  crlticlstn  by  educators  in  regard  to  use  of  behavioral  objectives 
is  that  it  takes  an  enormous  amount  of  extra  time  and  energy  to  formulate  and  use 
them.    Conroy  {1973)  points  out  that  most  conimerciany  produced  Instructional 
programs  thai:  do  provide  behavioral  objectives  describe  outcomes  that  are 
usually  limited  to  lower  level  skill  capabilities.    As  a  results  teacheri  are 
forced  to  write  most  of  the  objectives  for  the  programs.    Teachers  feel  that 
their  role  is  becoming  one  of  a  clerk  whose  role  is  to  mechanicany  grind  out 
behavioral  objectives*    Conroy  also  feels  that  even  though  there  is  broad 


agreement  about  the  benefits  of  using  behavioral  objectives  in  education, 
relatively  little  use  is  being  made  of  the  systematic  process  of  developing 
instruction 5  and  that  rarely  does  a  total  school  system  manage  its  entire 
instructional  program  by  behavioral  objectives.    He  concludes:    "The  fact  of 
the  matter  is  that  the  vast  majority  of  American  teachers  just  do  not  use 
behavioral  objectives  as  a  part  of  their  prrifesinonal  practice." 

^  Conclusions 

Although  there  are  several  logical  reasons  for  using  instructional  objec- 
tives, there  is  limited  empirical  data  to  support  their  usee    Kibler  et  aU 
(1974)  point  out  that  there  are  only  about  fifty  or  so  expun.H^rtal  studies 
focused  on  instructional  objectives*    Unfortunately ^  the  results  of  these 
studies  are  inconsistent*    They  also  provide  no  conclusive  evidence  about  the 
effect  of  Instructional  objectives  on  learning*    Kibler  states  -^of  the  thirty- 
three  studies  found  that  compared  student  learning  with  and  without  possession 
of  instructional  objectives  J'    Current  findings  on  the  effects  of  instructional 
objectives  provide  no  conclusive  or  consistent  data  on  the  relationship 
between  the  use  of  objectives  and  student  learning.    We  are  unable  to  draw  any 
conclusive  generalisations  about  the  effect  of  behavioral  objectives. 

It  was  the  purpose  of  this  paper  to  present  a  reviev/  of  the  literature 
concerning  the  definition,  forms  function  and  the  cases  for  an  against  tlie 
use  of  ..behavioral  objectives*    It  is  clear  that  there  are  many  different 
definitions  of  the  term  "behavioral  objective^'    Additional  development  is 
needed  in  this  area  so  that  a  consistent  operational  definition  of  the  term 
behavioral  objective  can  for  formulated*    Although  Mager's  criteria  for 
writing  a  well  stated  behavioral  objective  is  the  best  known,  literature  reveals 
that  there  is  little  agreement  as  to  which  characteristic  of  the  behavioral 
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objective  1s  most  important,  while  others  think  that  the  standard  or  condition 
is.    Gerlach  et  al.  (1977)  indicate  that  the  choice  of  direct  object  also 
influences  the  observability  and  precision  of  a  behavioral  objective.  They 
also  contend  that  no  single  component  1s  consistently  of  primary  importance 
in  determining  the  proper  form  of  a  behavioral  objective. 

Aside  from  the  problem  of  determining  the  correct  form  of  a  behavioral 
objective  and  the  need  for  training  educators  to  write  in  an  acceptable  form, 
the  literature  suggests  that  educators  must  also  be  presented  with  the  philo- 
sophy behind  the  concept  of  behavioral  objectives.    The  basic  characteristics 
of  a  behavioral  objective  are  not  fixed.    Educators  are  not  looking  for 
objectives  that  are  a  particular  size  and  shape,  but  are  looking  for  objectives 
that  are  clearly  stated  and  convey  our  Instructional  Intents  as  concisely  as 
possible. 

Although  there  are  disagreements  regarding  the  definition  and  form  of 
behavioral  objectives,  the  literature  reveals  that  behavioral  nbjectives  can 
have  important  functions  1n  the  Instructional  process:    (1)  aid  in  the  design 
of  Instructional  programs^  (2)  provide  guidance  in  evaluation  of  Instructional 
programs,  (3)  facilitate  learnin,  for  students,  and  (4)  Inform  teachers, 
administrators,  and  general  public  of  the  purposes  of  the  instructional 
program.    The  major  disagreements  lie  in  the  very  nature  of  the  behavioral 
objective  movement  Itself.    There  is  a  wide  disagreement  as  to  the  advantages 
of  using  behavioral  objectives.    MacDonald-Ross  (1973)  is  one  of  the  major 
critics,  stating  that  "behavioral  objectives  will  never  achiyve  all  that  their 
supporter  hope,  for  they  are  limited  by  the  very  presumptions  on  which  they  are 
based." 

As  an  advocate  for  the  use  of  behavioral  objectives  1n  education,  I  will 
conclude  with  the  following  impressions  that  I  have  drawn  from  review  of  the 
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literature^ 

1.  There  1s  a  need  to  generate  explicit  principles  for  constructing 
relevant  behavioral  oboectlves.    Rules  for  specificity  of  objectives 
need  to  be  developed. 

2.  Behavioral  objectives  form  a  well -worked  out  method  of  rational 
planning  1n  education.    They  encourage  educators  to  make  explicit 
the  values  they  may  have  never  revealed  as  well  as  making  them  think 
and  plan  in  detailed,  specific  terms^ 

3.  Behavioral  objectives  help  to  better  organize  the  students'  time  as 
v;el1  as  give  them  direction  and  provide  motivation. 

4.  Behavioral  objectives  can  form  the  basis  for  a  well-worked  out 
program  for  individualizing  instruction. 

5*  Behavioral  objectives  are  the  clearest  verbal  devices  available  to 
educators  for  use  in  communicating  the  intent  of  learning  programs 
to  students,  administrators ,  and  general  public. 

6.  Behavioral  objectives  can  provide  direction  and  guidance  to  teachers 
when  they  are  choosing  instructional  activities  and  materials  for  a 
learning  program. 

7,  Behavioral  objectives  provide  a  rational  basis  for  evaluating 
instructional  programs.    Even  with  the  high  "costs"  attached^  the 
cyclical  approach  to  evaluating  Instructional  programs  is  worthwhile. 
Good  Instruction  Is  not  developed  overnight  and  without  expense. 
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Abstract 


The  troubling;  Droblem  of  re^lRtlnrr,  refsearch  to  metlce  Is 
dlscuasfid  with  reference  to,  what,  why,  how.     Thn  wh^t  nf  rnnnnt 
resi^arch  Is  illustrflted  by  nxflrmlfts  whlnh  In-rnlotnrl 
researfth  la  beconln/y  moT-R  rlfl;orous,  analytlofll,  and  theoretloni , 
flnd  thus  more  reliable  and  .generally  applicable.     The  wh^  is 
briefly  argued  with  reference  to  the  greater  rellabl Uty  of 
reaearoh-based  Inf ormatlori  over  tlmt  l(»arned  on  the  Job  by  an 
Individual  practitioner.     The  How  Is  considered  by  notlhi.  the 
large,  available  reservoir  of  relevant  findings  and  by  deBcrlMn;? 
a  way  of  coffriltlvely  relating  the  practitioner's  questions  to  the 
researcher's  flndlnss  throuRh  their  common  Interest  in  relating 
Instrvotlonal  conditions  to  Instructional  effeots. 
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SELATING  BEHAVIORAL  SCIENOT  KSEARCH  TO  PRACTICE! 
WAAT,  VWY,  HOW 


Nine  years  ago  I  reperted  to  this  group  on  som  perceptual  principles 
for  the  c?esign  of  Inetrwtional  TOtarials.     The  oocasion  was  the  raeeting 
in  Detroit  of  what  was  then  DAVI*     I  nt  jased  five  principles  based  on 
perception  research i    Perception  is  VBi^y  selective,  is  influenced  by  a 
prior  set  to  see  Boim  things  and  not  otharSg  and  is  highly  organised* 
Further,  auditory  perception  is  specialized  for  temporal   ' '  tinccions 
and  visual  for  spatial  distinctions,  and  finallys  pictures  of  objects  are 
better  remembered  than  their  MTses,    Since  then  Dri  I^vte  and  I  have 
searched  out  numerQua  ather  principles  from  behavioral  science  research : 
56  from  the  perception  llteraturep  41  from  the  memory  literature,  27  from 
the  concept  learning  llteraturet  and  69  from  the  atti'^ude  change  literature. 
These  are  now  packaged  in  a  book  entitled  Instructional  MessaTO  Deslm  (1978)  * 
Consequently,  one  of  the  things  this  paper  will  not  Include  Is  a  reiteration 
of  any  of  those  research-based  principles  for  the  design  of  Instructional 
materials.     Instead,  I  will  address  the  general  problem  of  moving  research 
into  practice  and  do  so  from  three  perspectlwsi     the  what,  the  why,  and 
the  how. 

First,  the  what*    Obviously  it  is  Impossible  It   /hese  few  minutes  to 
deal  adequately  with  what  behavioral  science  research  is  being  reported  today 
and  what  part  of  that  is  particularly  pertinent  to  instructional  designers, 
developers,  and  media  wers.    SOp  1  will  briefly  deficribe  three  trends  in 
media- re  la  tad  research  and  give  a  faw  exa^lea  of  each.    Research  today 
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differs  from  tiiat  of  Verity  years  ago  In  several  important  waysi    It  Is  more 
rigorous 9  mora  analytical^  and  more  theoretical.    Althoush  these  characterlr: 
of  reBearch  my  fall  to  describe  the  applied  reaearch  which  many  profesaionai. 
praatlclaners  identify  wlthp  I  will  argue  and  give  eKantplaa  to  aupport  the 
opposite  vlew^  I.e.  that  It  la  the  rigorous,  analytlc^Wd  theoratlcal  research 
that  best  servos  the  practitioners  needs* 

Trend  nt^er  onep  more^rlgox.    Hie  Idea  of  rigor  is  slisply  that 
rtsearch  Is  batter  concaptMlliad    and  battar  controlled,  and  henca  raora 
reliable*     This  frequantly  lakas  th^.  fortn  of  replicating  prior  studies  or 
rs-addresslng  pravlouflly  studied  problams  with  Improved  procedures*  For 
example,  there  was  a  long-accepted  principle  that  knowledge  of  correct 
responses  (KCR)  would  be  reinforcing  to  the  learner  and  hence  would  be 
associated  with  increase  '  iMrnlng^    This  was  also     ^aslc  tew^  of 
progra^d  Instruction  (PI).    :atil  study  after  stud*      ..led  to  support  it* 
A  study  by  Mderson,  Kulhavy  and  Mdre  (1971)  took  a  tmt^  rigorous  approach* 
These  Investigators  noted  that  relRforaemlntj|wae  by  definltloiis  to  follow 
the  correct  reaponse  and,  further  that  the  response  was  to  be  made  to  the 
crlterial  cues*    Most  prior  PI  studies  had  not  controlled  for  these  factors* 
Subjacto  could  peak  at  the  correct  response  before  raspOTdlng  and  could 
readily  do  no  without  paying  much  attention  to  the  crlterial  cues*  Conse- 
quintly,  conditions  were  arrangedjby  way  of  cofflputar  terminals,  such  that 
eubjflcts  were  required  to  respond  before  receiving  KCR  and  were  variously 
encouraged    to  give  Qareful  attention  to  crlterial  cues*    The  results,  for 
once,  flhov^ed  a  slpilflMnt  difference  In  recall  favoring  the  KCR  condition 
over  the  no  KCR.    This  seemed  to  resolve  the  issue,  but  ^deram  and 
Kulha^  went  further,  they  repUcatad  ttia  study  but  inqluded  both  Peak  and 


No«'peak  conditions  in  a  controlled  Gomparlson*     The  **PeaW*  group  made 
higher  wlthin^-program  saoras,  but  the  No-Peak  group  mad*'  sifnlficantly 
superior  recall  Bcotm  after  the  program*    Increased  research  rigor,  ttius, 
produced  more  reliable  flndlngfl  which  are  of  direct  relevance  to  practttlon 

In    Mother  series  of  studies.  Levin  and  his  assoclatea  had  been 
studying  the  effects  of  two  treatroenta  on  children's  learning  of  prose 
stories.      Repeatedlyp  the  prose-plua^related'-plctures  condition  w^as 
superior  to  the  prose  alone,  the  storiea  being  read  aloud  In  both  caaes* 
For  investigators  interested  in  picture  effects,  these  raaujts  were  quite 
posltivep  but  rigor  required  that  they  consider  plausible  alternative 
IntBTpretatlons  of  these  data*    Oam  alternative  was  that  the  picturc^s 
Gontributed  nothing  imlquely  attributable  to  their  being  pictures,  for 
they  mi^t  be  oervlng  only  as  a  repetition  of  the  story,  which  repetition 
could  as  we^1  have  been  provided  verbally*    A  ^oliow-up  study  (Levinp  1976) 
included  a  verbal  repetition  condition  In  which  each  sentence  was  repeated 
aloud  elrter  by  the  narrator  or  by  the  learner,    Resultfl  showed  that  verbal 
repetition  increased  learning  over  i   -repetition,  but  the  single  verbal 
plus  pictm%^a  was  still  olgnlflcantly  superior  to  all  other  conditions. 

A  ^jore  general  sign  of  greater  rigor  in  our  field  is  the  mrked 
reduction  in  the  nuntoer  of  gross  media  comparisons,  for  such  studies  are 
ganerallv  both  conceptually  weak  and  procedurally  vary  difficult  to  control. 

Tread  nmaber  two,  more  analy tlcal.     The  most  apparent  IndlGatlon  of 
increafllngly  analytical  research  la  the  greater  numbers  of  Independent 
variables  esamlned  in  a  study,  i.e.  not  imt  redla  variables  but  also 
learner  imrlables  and  subject  matter  or  task  variables.    Tlmre  is  also  an 
increase  in  the  diversity  of  dependent  TOrlablea,  l,e*p  not  just  recall. 


but  fllao  recognition,  cencept  learning,  problem  solving,  attitude  change, 
and  phyaiological  changas.     Re^rding  learner  variables,  for  eKamplej  e 
greater  diversity  of  attributes  are  being  e^caminad,  e*g*i  sex,  co^ltive 
style,  developrent  level,    ^eults  inalude  much  e^/ldence  that  feimklea  are 
geneirally  more  skilled  in  verbal  tasks  than  iiales,  while  there  is  some 
avidenae  that  males  ars  generally  roore  skilled  in  some  spatial  tasks  than 
females . 

People  also  differ  In  important  waya  called  oo^itlve  style.  For 
e^i'^.n^le,  some  are  more  skilled  than  others  at  distinguishing  a  simpla 
visual  figure  from  a  eonpiax  background*    A  surprisingly  large  ainount  of 
research         been.  d:ie  with  this  basic  visual  perception  skill,  sora  of 
it  dem^v^=v:::.v.    liig  a  relationship  to  other  copiitive  skills  and  even  to 
social  dkilla*    This  s©--eailed  cog^.itlve  style  is  not  related  to  general 
intelligenca  but  does  seem  related  to  the  type  of  school  subject  and  type 
ef  iTistruction  rethod  that  learn/ars  can  most  readily  cope  with.  SubjeCi^a 
who  are  most  skilled  at  separating  figures  from  ootDplax  ^vuimds  (called 
fie  Id- independents)  tend  to  be  more  analytical,  prefer  Sciance  coxirsas  and 
mnd  working  alone s  and  ara  bettar  able  to  orgsfliizn  information  themaelves. 
Subjects  least  sklllad  at  figure--groimd  separation  (called  field-dependents) 
tend  to  ba  cognltively  mora  global,  prefer  socisl  studiae  courses  mi  working 
with  people,  and  Iwrn  bast  from  wall  or^nliad  learning  rflflterials  (Witkin,  at 
alt,  1979).    Thesa  more  analytical  findings  all  ha'^  intaraifclng  implications 
for  instructional  de^mlopers« 

Pictures  ire  also  being  studied  mre  analytically*     For  escampl^, 
Margaret  Hagen  (1974)  has  bean  studying        wa  parceiva  pictures  and  to  what 
a^tmt  and  in  what  ways  we  ^t^c  learn  to  percalva  them,  literature  la 


con tradlc Cory,  but  it  does  appear  tiiat  Infants  can  reco^i.^e  without 
training  a  pieture  of  a  fimlliiX'  objrGt«    However,  skill  in  perceiving 
certain  aspects  of  cofflpleK  picturoa  must  apparently  be  learied.  For 
e^Tample^  we  must  learn  to  adjust  to  impoverished  information  '3#gp  lack  of 
carrain  c^ent-H  euas  in  flat  pictures,  and  lack  of  motion  cues  in  still' 
pictyre*..  '        .^st  learn  to  intek^yelatd  various  shades^  teHtures^ and  forms  so 
as  to  tadi^a  their  .^ambiguity  *    Jean  hfandler  (19  76)  has  studied  the  kinds  of 
iaforration  which  people  selectively  remember  in  pictureai    Evidence  to  date 
suggests  that  the  identity  of  objects  in  a  pieture  is  remembered  bastp  the 
spatial  location  of  tlioee  objeeta  somewhat  less  well,  md  the  interior  details 
of  the  objecfco  least  well.    Needless  to  sayp  these  analytleal  studies  are 
highly  rele'ront  to  profesolonal  practiae  in  our  field* 

Analytical  studies  produce  quantities  of  findings  which  are  compli^Kly 
ralmted  to  nun-f*^roufi  variables.     Ttm  effect  is  to  greatly  complicate  the 
task  of  Interrelating  such  studies  so  as  to  SOTaari^Sf  generalise  and  form 
some  uacful  conclusions.     It  is  here  that  overriding  conceptualisations, 
models^  and  theories  are  indispensable. 

Trend  number  three p  more  theory.     Theories  are  not  only  more  general 
than  indi%        .1  bindings,  they  also  explain  why^  and  thus  are  of  great 
potential  t 

Although  nwoneroua  theories  have  been  genera  teds  it  is  still  not  clear 
why  pictwo'es  are  more  memoreble  than  words,  but  the  effort  to  explain  this 
general  finding  has  prodi'     ^  much  Important  research.    For  exanqple,  several 
conditions  have  been  identified  in  whic'    pictwes  are  not  superiors  s*g*i 
where  the  plctyres  are  presented  at  a  rapid  rata  and  also  where  they  are 
quits  similar  to  each  other. 
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Otim  of  the  bast  supported  theories  is  dual^coding,  I.e.,  that  pictures 
are  wotm  meodrable  becawe  they  are  more  likely  to  be  coded  tt^lce,  both 
lioaglrally  and  verballyy  and  dually  stored*    Words  are  less  meniorable 
becauaa  chey  are  less  likely  to  be  dLwlly  coded  and  stored.    Howevetp  It 
la  clear  that  both  are  readily  recodadp  i.e**  v?e  see  a  picture  of  a 
friend  and  think  of  his  name  or  we  hear  or  read  hi,B  name  and  think  of  a 
mntal  picture  of  his  face.    The  consequences  of  this  for  msdla  professionals  are 
potentially    shattering^  for  the  cognitive  affecti  of  our  stlraMll  are  not  given 
by  ^helr  objective  form^'^the  effect  of  a  concrete  picture  My  be  an  Abstract 
verbal  concept.    Like  It  or  not,  these  and  other  theoretical  Insights  will 
change  our  professional  practice. 

Dual-^coding  theory  is  similar  to  others  in  which  eKplanationa  are 
sought  at  the  processing  level  through  such  questione  as  ^    How  are  eKtamai 
stimuli  T^p4"eaented  internallyp  how  are  the^^  .i^angad/trmnslated/elaborated 
durini?  such  processingf  how  are  they  stored  in  memory i  and  haw  ara  they 
^  -  ^     rs4£a4w4  and  re^p resented  to  awareness? 

A  con tt:;.iporary  theoretical  accoimt  is  called  depth-of --procesfiing,  i.e., 
the  greater  the  depth  (or  breadth)  of  processing  of  a  picture  or  word  the 
greater  the  recall*    For  example*  Bower  and  Karlln  (1974)  ^ound  significant 
diff^reuQes  in  recopiltlon  memory  for  pictures  of  people  depending  on  the 
depth  of  processing  prasumbly  induced.    Subjects  instructed  only  to  judge 
the  sex  of  the  pictured  people  reMrabered  the  pictures  less  well  than  those 
instructed  to  judge  the  honr    /  or  the  llkeab.\enaaa  of  the  people,    toe  of 
tiia  implications  for  mssage  designers  is  that  the  mere  presentation  of 
nicturea  la  not  enough #    The  learner  must  be  furthtr  involved  in  fttudying 
thda  (by  way  of  introduction  or  task),  in  attending  to  the  most  critical 
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aspceta  (by  way  of  polnterSp  capclona)^  in  thinking  ybout  them  (by  way  of 
answering  questions,  solving  probletns).     This  theoretical  reaeafeh  thus 
yiajds  motiier  prastlQal  Inflight i     the  leanier  must  procesB  information  In 
mtm  tfian  superflaial  waya,  md  can  be  influenced  to  do  so  by  appropriate 
instructional  design. 

HieoretiGally  ©rlented  research  on  visual  proeefislng  has  produced 
*u*  ny  veiy  Interesting  and  releTOnt  insighrs  for  our  field,    llie  mental 
rapresentatlo*!  of  a  plcturip  object  or  event,  t.m,  a  picture  in  the  ndndp 
has  been  shown  to  participate  In  nuonrous  cognitive  proceaies.    A  rrantal 
image  can  rarkedly  faeilitate  memo^  for  both  words  and  pictures  (Palvio,  19 
It  appeara  to  aontribute  to  concept  formation  by  rep'resentlng  the  most 
cotmaon  or  pTOtotyplc  ea^mplas  of  the  concept  (Roschy  1978);  it  appears  to 
function  importantly  in  the  solving  of  spatlai  problems  and  of  spatial 
m^B  like  chess  (Chase  and  Simon,  19  73);  It  la  widely  iaportad  to  have 
been  the  inedium  of  crmtlva  and  jta'rontive  thought:  used  by  both  artists 
and  sclentlatsp  Including  Einstein,  (Shepardv  1978)*     There  is  also 
evidence  thai  thinking  visually  la  a  cagnitlva  skill  that  ran  be  developed 
through  repeated  experirnce  with  vlai^l  medlp  such  as  film  (Salomon,  1978) • 
Tile  Impllaatlons  of  suoh  theoretical  wort;  on  mental  Imagery  are  mind 
boggling,  eapeclally  for  visual  madia  rebearehers  and  prac titlonerfl • 

In  sump  1  have  endeavored  to  indicate  that  medla--related  research  has 
become  Increasingly  rlgoroi^,  analytical,  and  theoretical,  and  that  the 
effect  for  profesalonal  practitioners  has  been  both  an  Increased  reliability 
md  a  broader  generality  of  eor  luslone.    So  much  for  the  uhat  of  relating 
research  to  practice* 
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Thm  why  of  relating  reeearah  to  practice  my  nfeed  nn      M.*u>-flim  for 
this  grr»app  it  being  imd.grstood  that  a  ptofeaai™  la  cf:%mct^TlmC  bv  i^uft 
reliability  of  ita  knowledge  and  the  relevanca  of  that  knowledi;^  fco  im 
practieep     If  our  research  does  no'  infora  our  praatlca  thefe  is  little 
juatiflQation  for  our  profesBlon  to  engage  in  it#    Howaverp  one  nan  askj 
^"Wliy  vQthp    relating  resMrah  to  practics  where  a  seasonad  profe8Bit;rvs;l 
vlli  have  discovered  mny  of  the  anflwers  simply  by  facing  a  variety  of 
pfoblema,  trying  a  wriety  of  solutionsp  and  observing  the  rfisulta?^'  toe 
argymant  for  research  even  under  these  cireumstances  la  that  the  practitioner 
prlnarily  learns  from  his  own  individual  experience  while  the  researcher 
generally  builds  upon  the  accumulated  knowledge  of  numeroua  other  Investi^ 
gators*    Further,  researchers  generally  are  inore  precise  in  setting  up  the 
''test'*  situation  and  In  analysing  and  evaluating  che  ffesultg;  and  they  more 
regularly  go  public  with  their  resulta  so  that  others  can  criticise  them 
anck  di^vBlop  other  studies  to  repeat  or  contradict  the  findings.    For  these 
several  reasons  research  tends  to  produce  raore  syeteTOtic  and  reliable 
Inforaatlon  than  prautica^  at  least  In  those  dominB  of  hunan  eKpvc  rience 
that  are  aMnable  to  research  raethodolo^.    But  in  addition  to  reliability, 
research  must  also  pass  the  teat  of  relrivfince  before  It  can  be  successfully 
related  to  practl.;*i.    It  is  hsre  that  my  third  topic t  the  ho^  of  it,  eomes 
can  research  be  made  relevant  to  practice? 
My  answer  assuc^s  that  the  research  In  question  Hbe  been  or  will  be 
well  done,  l*e*  neets  the  criteria  previously  descrlbadt     rigorous  * 
Analytical,  and  theoretical*    Ity  answer  also  assumes  that  the  research 
la  potentially  ireful  to  media  practitioners,  and  that  Includes  a 

suiqprlaljigly  large  proportion  of  the  very  extensive  body  of  research  on 


instruetloni  laarningp  and  eomimieation*     OlLvn  a  practitioner  iMking  a 
film  on  aarth  aGience  for  eighth  graders »  far  «iixample|  feels  that  the  only 
relewnt  research  Is  that  on  films  for  eighth  grade  science*  Aatually 
many  areas  of  reeeareh  may  be  per;.inenti    how  eighth  graders  learm  In 
generalp  the  effect  of  ttielr  stege  of  cosiitive  developMnt^  and  their 
Intereats;  the  characfterlstlea  of  scienM  aa  a  form  ©f  knowledge,  the 
differaneea  between  learning  facts ,  ^-onceptss  and  skills |  the  potential 
strengths  and  weataiesBes  of  film  in  Instruction,  the  fixed  pace,  the  lack 
of  learner  crotrol.    Also  pertinent  la  the  research  on  the  many  stimulus 
eleifisnts  available  la  film  such  as  motion,  color,  pi0ture8  vs*  worda, 
picityrea  interacting  with  words,  the  sequence  or  order  of  pictures  mid  or 
words,  the  characteristics  of  visual  perceptual  channel  vs*  audito]^ 
channel,  eta,,  etc*    The  extensive  research  on  all  these  factors  assures 
chat  there  is  no  dearth  of  potentially  relevant  research  findings  for  the 
'^.arth  science  film  maker  or  most  any  other  Inatructlonal  dr^\feloper# 

Professional  practice  in  our  field  la  frequently  represented  by  a 
modf  l  which  depicts  a  series  of  steps  -^r  operations  which  the  prec tifiion^r 
must  perform  or  supervise.    Research  la  seldom  depleted  in  chese  pro^^dMil 
models,  though  it  has  son^times  been  suggested  as  somehow  Lmderlying  the 
proceduras  or  surrQimdlng  them,  the  idea  being  that  practice  somehow  takes 
place  in  a  nutrient  Tiediura  of  research.    But  the  raneral  absence  of  reference 
to  reB^,areh  in  development  models  mi^t  be  n^^lalned  in  some  way,  e*g*, 
research  is  considered  irrelevant  or  only  retnotely  islated,  or  research  is 
assiflned  throu^out  and  hence  not  di.piraed,  or  knowledge  of  research  (as 
with,  other  forsB  of  knowled|e)  doestv  i:  beJong  in  m  Intrlniiically  jrocf-dural 
model*    I  prefer  the  latl,eT  Intoif^tatat^ioti,  that  .nnmmh  repres^Pf:^  an 
esseatlAl  but  nOT^pro^duraA  wpfesst  ox  ttm  process* 


While  there  Is  Important  reaeareh  on  development  procedures  thprnselves. 
It  la  1^  vltw  that  moat  eurrent  research  pertinent  to  design  and  developinent 
ifflptages  on  the  way  in  which  profeastorials  COTceptualize  or  think  about  the 
procesa*    What  kinds  of  thought  processes  are  most  produetlve  of  viable 
Bolutlona  to  design  and  davelopaent  decisions?    Da vies  (19  71)  and  others 
have  written  about  the  process  of  decision  roaklng*    My  concern  is  with  the 
substance,  the  oontentp  of  the  deeision-raakers  thought  pr.     ^    \  ,  £ot  It 
is  here  tha^  researah  convergeg  on  practice. 

Probably  the  question  most  central  to  the  developer's  t.ifeughts  la  i 
What  Instructional  condltlona  will  Mat  effp.stively  and  efficiently  guide 
the  learner  to  the  competMoe  desired?    This  reduces  t©  the  basic  relation 
between  inatructienal  conditions  and  laarnlng  effects,  or  more  fimdamgntally 
to  the  relation  between  conditions  and  effects.    Both  research  and  practice 
can  idi atlfy  with  this  relationship.     The  researcher^s  interest  la  In 
studying  the  relation  between  conditions  (the  independent  variables)  and 
effects  (the  dependent  TOrlables)|  while  tiie  practitioner's  interest  la  In 
deciding  whidi  condition  will  inest  likely  produce  the  effect  he  dealres. 
It  Is  at  this  coifffnonly  held  basic  conceptual  level,  relating  conditions  to 
effects,  that  I  believe  the  rapprochement  be^een  rssearch  and  practice  can 
most  appropriately  be  Mde.    The  conOTOnallty  that  makes  rapprochement 
poaslble  can  be  representsd  by  the  parallel  forms  of  two  statements:  one 
the  researcher's  conditions  and  the  otl  ^ir  the  dii^loper*s  quastlons* 
Baae&rchars  cCTicludel 

Certain  oonditlpns  have  lead  to  eergain  ef feota* 
Developera  ask: 

<^^c  conditlone  will  lead  to  the  desired  effects? 


Clearly  tha  juKtJipQslcion  of  the  developer's  queationa  with  the 
appropr^ce  researcher's  coaalualons  has  the  effecc  of  relating  reseerch 
tn  praetlee  in  a  dir^f^t  way,    A  designer/developer  who  conaeptualiMs , 
thinks  aboy  t»  and  di^ir  usses  dealflions  in  this  way  is  more  likely  to  Mka 
raliabla  deelflionF      .      ^cuiarly  if  he  has  in  mind  an  adequata  supply  of 
raliabla  etatemaata  linkini  conditions  to  effectOt    Tlieie  linking  statemfentfl 
I  call  instructional  deslo-i/develQpMnt  prtaclplei.    Exacples  are  the 
pereeptiDn  prlnaipleij  cited  at  this  aonventlon  almost  tan  years  ago  and 
the  nuuseroys  others  cited  misicm* 

Lest  anyone  think  this  eopiltiw  Mtching  of  research  findings  and 
practitioner  questions  is  so  pat  a  process  as  to  autOMtically  produce  a 
correct  solution  oust  realise  that  the  principles  do  not  apply  to  all 
design /development  problems  and  even  where  applicable  sarve  only  as  fairly 
raliablB  gyidss*    Thsy  musE  always  b#  tested  in  each  design/development 
applicationp  l#e* p  prototypie  nmasages  must  be  given  careful  formative 
evalu#  .  Ion* 

In  i^im  I  have  endeavored  to  report  a  .Tew  examples  of  recen;:  research 
in  our  field  that  are  increasingly  rigyroua*  analytical*  and  theot^itlcal, 
and  thereby  are  more  reliable  and  generally  useful  for  professional 
instruefiional  designers  and  dewlopers.  Further  I  have  provided  arguments 
why  muah^  research  in  Instruetionp  Isarningp  and  eomiimication  is  relevanf- 
to  our  profissional  practi-.*  and  have  daocribed  a  co^itlve  process  modal 
for  relating  research  findings  to  daelgner/developera  questions. 
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ABSTRACT 


This  pap^f  describes  a  model  for  the  systematic  designs  developnientj  and 
validation  of  altematiye  instruational  modulesf  by  incorporating  the  basic  coneapts 
underlying  ATI  analyseSi  The  proposed  working  model  is  based  on  tlie  premise  that 
the  ATI  oonceE^t  Is  best  applied,  not  in  isolation^  but  as  an  integral  part  of  a  dynamic 
daeision  making  instpuctlonal  environment.  Four  areas  of  interestj  for  applying  the 
ATI  eoneept  in  an  operational  environnnenty  are  diseussedi  (1)  Operational 
Methodoloo?  (3) 'Variable  Categoriesi  (3)  Statistical  Proceduresi  and  (4)  Alternative 
Module  Developmerit  Guidelines* 


a: 


APPLYIHG  THE  ATI  CONCEPT  IN  AN 
OPERATIONAL  ENVIRONMENT 

Overvi€w 

For  many  yearSj  niueh  has  been  written  about  ATI  methodology,  ordinal  and 
disordinal  interactions,  and  the  search  for  meaningful  hypothesei  involved  with  ATI 
research.  In  this  brief  paperi  the  writer  has  attempted  to  synthesize  both  prior  work 
experienee  and  prior  readings  into  a  working  rnodeL  This  nnodel  attempts  to  set  forth 
a  framework  for  designing  alternative  instructional  modules  that  have  revvards  for 
both  the  learner  and  instructor* 

An  aptitude  variable  can  be  any  personological  or  organismle  variable  upon 
which  individuals  differ  (e,g*  IQ,  anxiety,  dogmatism,  etcJ<  A  treatnient  is  any 
instruetional  strategy  or  combination  of  instruetional  strategies  that  structures 
information  for  the  purpose  of  having  students  learn  that  informatioa  As  generally 
defined,  an  aptttude-treatnient-lnteractiQn  exists  when,  as  a  result  of  a  given 
treatment,  individuals  at  one  end  of  an  aptitude  variable  perform  at  one  level  on  a 
criterion  measure.  Individuals  at  the  other  end  of  the  aptitude  variable  perform  at  a 
significantly  different  level  on  the  criterion  measure  and  the  reverse  trend  holds  true 
for  a  second  treatment*  (For  an  expanded  discussion,  see  Snow  and  Salonionj  1968; 
Bracht,  1970^  Rhetts,  1974;  Snow,  1977;  Cronbaeh  and  Snow,  1977*) 

The  literature  is  filled  with  plots  (varying  greatly  in  degree  of  angularity)  of 
different  treatment  groups  against  both  aptitude  and  criterion  measures*  However, 
little  if  anything  is  written  which  proposes  how  to  incorporate  the  ATI  concept  into  a 
decision  making  instructional  environment.     Two  common  oriticisriis  of  the  ATI 


apppoaeh  are  that  generalizations  resulting  from  ATI  research:  (l)  Uck  a  design 
strategy  that  demonstrates  a  need  to  develop  alternative  modules  in  thfi  first  placej 
and/or  (2)  are  not  easily  interpreted  so  as  to  provide  detailed  gui^ellnos  for 
instructional  desig'ners.  The  proposed  working  model  is  based  on  the  prerriJse  tli&t  the 
^TI  concept  is  best  applied^  not  In  isolation,  but  as  an  integral  part  of  \t\e  dynamic 
decision  making  instructional  environment. 

Four  areas  of  Qonsideratlonj  for  applying  the  ATI  concept  in  an  cperational 
environment^  will  be  presented:  (1)  Operational  Methodology  (overall  master  plan)  (2) 
Variable  Categories  (for  independent  and  dependent  variables)^  (3)  Statistical 
Procedures  (analyzing  the  results)^  and  (4)  Alternative  Module  Development 
Guidelines  (detailed  guidelines  for  instructional  designers)p 


Underlying  the  innple mentation  of  this  model  is  the  assumption  that  for  wont  of 
us,  the  time  and  expense  involved  in  alternative  module  development  is  not  \vorth 
taking  unless  the  existing  instructional  module  causes  large  or  alarming  vafiations  in 
the  criterion  variable.  That  is^  if  everyone  is  reaching  criterion  or  achieves  mastery 
(passing  grade)  in  an  aceeptable  time  frame  within  a  given  module^  why  u&e  resources 
searching  for  ATrs  or  developing  alternate  modules?  It  Is  only  when  gradas,  time  to 
completion,  or  some  other  dependent  variable  of  interest  and  inriportance  varies 
within  a  given  module,  that  educators  should  be  concerned  about  developing  ^tjrnate 
moduleSi  Only  then  does  it  seem  justifiable  to  allocate  resources  for  alternative 
module  development. 

The  operational  methodology  suggested  here  involves  five  main  st^ps  (refer  to 
Figure  l)i 

Step  li       Establish  Main  Track  ModuleCs)  (Point  "A'O 


Establish  a  single  or  main  track  of  instructioni  that  is,  develop  a  ^^best 


f  uess"  approaeh  by  designing  one  module  per  instructional  concept  for  the 
entire  unit  or  course  being  taught. 

Step  2^      Establish  Data  Analysis  Pool  (Point  "B") 

Establish  a  data  pool  of  variables  of  interest  and  importance.  These 
variables  must  be  onei  that  are  expected  to  be  olosely  associated  with 
performance  on  tlie  criterion  measure. 

Step  3t       Asgess  Varianee  (Point  ^'C'') 

Examine  the  dependent  variable  of  interest,  usually  time  or  score,  with 
re|ard  to  "annount  of  varianee  in  main  track  instruction.  Identify  main 
track  modules  with  unacceptable  varianee  in  the  dependent  variable. 

Step  4:       Analyze  Data  (Points  "D"  and  ^'E'O 

Analyze  data  and,  based  on  significant  predictor  set(s)s  develop 
alternativa  modules  for  main  track  modules^  only  where  large  variations 
exist  in  the  dependent  variable. 

Step  5i       Validate  Alternative  ModuleCs)  (Points       and  "G") 

Validate  alternative  module  development*  Examine  actual  perforrnanoe 
data  assaoiated  with  predicted  performanGe  data  under  varying  module 
assignment  rulesi  i.e.,  randoni  vs.  non-^random  for  both  main  track  and 
alternative  module(s).   Assess  results  and  make  decision(s). 


Variable  Categories 

Once  the  main  track  modules(s)  have  been  developed  (Step  1,)  serious 
consideration  must  be  given  to  the  selection  of  variables  to  be  entered  into  the  data 
pool.  At  least  four  eategories  of  independent  variables  are  suggested  for 
consideration  during  Step  2  (Figure  1,  Point  "B^')i 


Cognitive 


2, 


Achievement  Motivation 


3. 


Personality /Anxiety 


4, 


Derived 


COGNITIVE  -  This  v^ould  be  either  a  general  mental  ability  measure  or  an 
aspect  of  general  mental  ability  for  which  a  strong  rationale  could  be  built  that  this 
aipect  is  related  to  performanee  on  the  dependent  variable  (e.g,  high  spatial  ability 
skills  required:  measured  by  performance  on  a  visual  discriniination  test), 

ACHIEVEMENT  MOTrVATION  -  This  element  of  the  data  pool  is  extremely 
important  in  determining  the  degree  to  which  ar  individuars  motivation  to  succeed  is 
related  to  the  task  at  hand.  This  variable  must  be  desifned  to  measure  ^eeific  task 
achievement  motivation  for  the  dependent  variable  of  interest  (e,g,  high  suecess  drive 
required:  measured  by  performanee  on  a  test  to  identify  potential  corporate 
executives)*  For  an  excellent  discussion  of  factors  to  consider  in  the  selection  of 
this  type  of  variable,  see  deCHARMS  and  UVIE  (1978), 

PERSONALITY /ANXIETY  -  According  to  Cronbach  and  Snow  (1977)  analysis  of 
achievement  via  levels  of  anxiety  is  a  potentially  useful  concept  that  has  emerged 
from  analysing  many  ATI  studies-  The  optimal  level  between  anxiety  and  various 
levels  of  state/trait  curiosity  as  predictors  for  differing  kinds  of  performance  tasks 
has  yet  to  be  widely  studied,  much  less  reported  in  detaih  Among  those  who  have 
written  on  this  topic  are:  Leherissey,  0*Neilj  and  Hansen  (1972  and  1973)i  Spielberger 
(1972);  and  McCombs  (in  press), 

DERIVED  -  A  fourth  category  of  Independent  variable  that  may  deserve 
attention  in  more  sophisticated  Instructional  environments  is  a  category  often  called 
a  "within'-course*'  or  "derived^-  variable*  It  is  simply  using  time  on  one  module  (or 
some  other  variable  whose  value  is  not  determined  until  after  the  start  of  an 
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instructional  unit)  to  prediot  time/score  (or  some  other  dependent  varieble  of 
interest)  on  subsequent  modules.  This  set  of  variables  is  often  only  able  to  be  studied 
in  very  sophisticated  computer  managed  systems  where  dynamic  decision-making  and 
data  eolleetion  are  possible. 

Dependent  variable(s)  must  also  be  selected.  These  are  usually  either  time  to 
eompletionj  number  of  trials  to  Gompletionj  total  scores  or  some  combination  of  all 
three* 


There  are  basically  two  stages  in  the  data  analysis  procedures.  They  involve* 

1,  Identifying  the  main  track  modules  with  large  variations  in  the  dependent 
variable  (Figure  1^  Point  "C")* 

2.  Using  stepwise  multiple  regression  to  identify  a  predictor  set  that  is  predictive 
of  dependent  variable  values  (Figure  1,  Point  "D")* 

ASSESSING  VARIATIONS  -  Admittedly  identifying  main  track  module(s)  with 
excessive  variations  in  the  dependent  variable  is  often  an  arbitrary  decision*  There 
are  many  iophisticated  statistical  techniques  to  determine  empirically  what 
constitutes  large  or  significant*  However,  the  emphasis  here  is  on  presenting  a  model 
or  procedure  that  stresses  operational  eonsiderations. 

Preliminary  examination  simply  of  ranges  standard  deviation  and  mean  score  for 
each  main  track  instructional  module  dependent  variable  can  provide  a  wealth  of 
usable  information.  Each  educator  must  then  make  some  hard  decisions  about  what  is 
aceeptable  performance  variability*  When  modules  have  unacceptable  or  lai^e 
variations  in  penformance,  they  then  become  candidates  for  further  and  more 
discriminating  statistical  analysis.  Where  large  variations  in  main  track  modules  do 
not  exists  it  only  makes  good  sense  to  either  continue  using  that  module  or  make 
minor  revisions  and  repeat  the  analysis  process. 


Statistical  Procedures 
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DETERMINING  PREDICTOR(S)  -  Once  a  module  haa  been  identified  as  having 
unaeceptable  variations  in  the  dependent  variable  (that  iS|  some  sub-group  of  students 
performed  well  or  quickly  while  another  iub-fpoup  of  students  performed  poor  or 
slowly),  the  task  of  determining  the  best  predictors  of  performanee  begins. 

The  analysis  procedure  suggested  here  is  to  use  stepwise  nnultiple  regression  to 
identify^  from  the  previously  obtained  data  analysis  pool,  significant  predictor  set(s) 
for  the  performance  variable.  By  examining  the  compleK  nature  of  significant 
predictors,  a  more  accurate  estimate  of  those  variables  ^eGificaUy  associated  with 
performance  by  a  sub-^roup  of  students,  when  attempting  a  given  taskj  is  possible. 
Laoking  a  theoretical  framework  for  the  selection  of  variables  to  be  entered  into  the 
analysis  pool  would  be  a  major  deficiency  in  the  implementation  of  the  procedures 
suggested  here* 


Once  the  predictor  set  is  identified,  alternative  module  development  begins 
(Figure  1^  Point  *'E'0*  For  illustration^  assume  the  following  hypothetioal  situation 
exists  when  attempting  to  predict  a  score  on  a  particular  main  track  module. 


Alternative  Module  Development  Guidelines 


Predictors 


Simple  r 


Mult  R 


Curiosity 


+  .40 


AO 


.16 


Spatial  Ability 


+.38 


Verbal  Reasoning 


.72 


.52 
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In  this  situation,  we  see  that  curiosity  and  ^atial  ability  are  positively  related 
to  performance  while  verbal  reasoninf  is  negatively  relatedi  alsoy  52  percent  of  the 
variance  in  the  dependent  variable  is  accounted  for  by  combining  all  three 
independent  variables  as  predictors,  One  instructional  strategy  that  might  be  worth 
pursuing  would  involve  building  an  alternative  module  that  aroused  curiosity  while 
compensating  for  lack  of  spatial  ability.  This  alternative  module  would  be 
hypothesized  to  be  appropriate  only  fop  low  verbal  reasoners.  Such  a  module  might 
have  advanced  organizers  that  demonstrate  how  the  material  being  presented  is 
related  to  future  job  skills  or  arouse  curiosity.  The  instructional  content  would  then 
use  few  or  relatively  simple  visuals  (compensate  for  spatial  ability).  Againj  this 
module  would  be  designed  solely  for  low  verbal  reasoners* 

The  above  instructional  strategy  is  not  conclusive.  Howeverj  it  is  a 
researchable  strate^  that  is  meant  to  be  (l)  empirically  derived  and  (2)  based  on  a 
rationale  to  facilitate  learning  for  a  group  of  students  who  might  be  able  to  profit 
from  some  kind  of  alternative  module. 

During  the  alternative  module  validation  phasej  regre^ion  analyses  would  be 
runi  against  the  criterion^  on  various  combinations  of  main  track  and  alternative 
modules  whioh  had  been  assigned  to  (1)  aU  students^  (2)  predetermined  sub=groups  of 
studentSj  and  (3)  random  subgroups  of  students.  By  comparing  these  regre^ion 
slopes,  and  weighing  the  cost  Involved  in  alternative  module  development  against  the 
payoff  derlveds  decisions  could  be  made  about  whether  to  continue  alternative  module 
implementation  or  continue  with  main  track  materials  only  (see  Figure  1,  Points  "F" 
and  "G*0- 


ERIC 


If  the  validation  phase  bears  out  the  proposed  hypothesis,  then  the  alternative 
module  would  be  assigned  to  this  subgroup  of  students  as  the  ''primary"  module  at  the 
start  of  the  instructional  unit.  It  Is  in  this  area  of  module  assignment,  module 
account ingj  and  record  keeping  that  computer  managed  systems  are  most  valuable* 

Summary 

The  above  procedure  is  an  attempt  to  synthesize  some  of  the  recent  thinking 
about  ATI  studies  and  alternative  instructional  module  development.  After  allj  ATI 
analyses  and  resulting  alternatives  are  neither  always  appropriate  nor  cost  effective 
and  should  be  viewed  as  part  of  a  larger  picture*  That  is,  thr  should  be  viewed  as 
part  of  an  empirically  based  decision  model  for  alternative  instructional  module 
development. 
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Figure  1  ■  A  Data  lased  Procedure  for  Developing  Alternative  Initructional  Hodules 
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ABSTRACT 


Et       becoming  LTX^cr^asitigly  ^vldetic  chat  th^re  are  mariy 
Saf^ciexicles  in  tTi^  Uemrchlc^l  taslc  emlysis  ^ppraaGLh  to 
cSaB^-gnltig  instrac  tioti,  El^totatLo^  theory  of  Irtsttuc- 

ciornwas  develop  ad  aS        gXt^rnative  tk^t  o^e^cpmee  tttos^ 
aDficiencies.     TU  ^  %  ht^om^ioTi  Tbeotrf  a^lla  tor  bagtmlng 
the   instruction  >rdth  a  special  TcLTid  c^f  ^vervl^w^-one  tTiat 
»pieonilzes  the  iTi^tr^iccipn^l  content  ^ath^^  ttiari  siiMia^izlRg 
i_t.     Then  it  call^  fiCr  eLabor^titig  ^xt  ttat  over\rle>7  ia  a 
&  pec?ifl.c  way^^by  adding  dleeatl       torapl^xAty  in  "Laycr^-* 
a..er^ss  tlie  entire  br^adt^  c?f  the  content,   one  layeir  at  a  ^ 
t  im»j  until  the  desl^a^  Le^reL  of  detail  coi^pLe^lty 


J.  0 


In  SeaTch  of  a  Bettar  Way  to  Organize  Xnstruction: 
The  Elaboration  Theory 


The  Elaboration  Theory  of  Instructlori  is  ati  alternative  to 
the  sta:nda^cl         o£  organising  Ins  true  tion  based  on  a  hierar- 
chical taslc  analysis.    The  hierarchical  organtzatiori  results 
in  an  Itistructional  sequance  that  begins  VJtth  highly  frag- 
mented^ small  pieces  of  the  suhject-matter  contenit,  l<any 
educato:cs  have  found  the  fragmentation  to  be  demo t Iva t Ing  ^ 
Many  ediicatlonal  psychologists  have  found  the  parts^to-io'hole 
sequence  to  be  inconsi stent  with  much  knowLedge  about  how 
ieartilng  occtirs  most  eff ectively^^namely  schetaa  tbeory  and  its 
predecessor,  sijbsumption  theory^    And  itiany  Instructional 
designearp  have  found  that  1  earning  hierarchies  represent  a 
very  IncQb^pLate  basis  upon  ^^hich  to  make  d&cisioas  about 
sequencing  the  instruction,  mostly  because  learnltig  hierar- 
chies are  oaly  one  aspect  of  Che  structure  of  subject-matter 
content.    All  this  is  not  to  deny  that  learning  prerequisites 
eKtst  nor  to  negate  that  they  are  important --tliBy  do  exist 
and  they  ate  liaportant*    Rather  this  affirms  tliat  learning 
prerequisites  are  not  a  sufficlGiit  basis  for  orgatilEing  a 
whole  coarse  I  our  knowledge  must  progress  beyond  the  hierar-* 
chy. 
Context 

Instructional  design  theory  Is  concerned  wltli  methods  of 
ins  true  tiqtii     It  Is  helpful  to  conceptualize  tw  different 
levels  of  methods  for  organising  instruction:  nlcro  strategies^ 
-which  ar©  methods  for  organiaing  the  instruction  on  a  single 


topic  (l,e^^  on  a  singie  concept^  principle,  etc,)^  smh  as 
generalities,  examples,  practice,  and  feedbacki  and  macYO 
strategies^  which  are  methods  for  organising  those  aspects 
of  Instruction  which  relate  to  more  than  one  toplc>  such  as 
sequencing  (ordering)     synthesizing  Cintegrating) ,  and  sxinmax- 
izing  (previewing  and  revieming)  all  of  the  topics. 

The  Elaboration  Theory  of  Instruction  is  a  partial 
theory  of  instructlon---lt  does  not  atte^mpt  to  deal  mth  all 
aspects  of  instruction.     It  does  not  deal  with  mioyo  strategieg 
for  ors^i^i^lng  instruction^  although  it  can  be  and  is  being 
eKpanded  to  include  such  strategies, (Ralgeluth  &  Merrill , 
in  preparation).     For  a  good  dBscriptlon  of  inlcrD  stratagieSj 
see  Merrill *s  component  display  theory  (Merrill,  Ueigeluth, 
&  Faust,  1979;  Merrill,  Richards^  Schmidt,  &Woo41977)* 
Also,  the  Elaboration  Theory  does  not  attempt  to  deal  mth 
strat  :;-les  for  delivering  Instruction  (e.g*,  media  aelectlon), 
nor  does  It  deal  with  strategies  for  managing  instructioii,. 
Finally^  most  aspects  of  strategies  for  motivating  students 
are  not  included  within  the  present  domain  of  the  Elabora-^ 
tion  Theory*    But  all  of  these  aspects  of  instructional  theory 
will  ha  integrated  with  the  ELahoratlon  Theory  In  the  fore- 
seeable future.     The  Elaboration  Theory  of  InstructloTi 
presently  deals  only  i^lth  macro  strategies  for  organizing  in- 
struction (see  Figure  1) , 


Insert  Figure  1  about  here 
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An  Atialogy 

A  good  introduction  to  the  nature  of  the  Elaboratipn 
Theory  of  Instruction  is  an  analogy  with  a  zooxn  lens. 
Taking  a  look  at  a  subjeGt  matter  "through"  the  Elaboracion 
Theory  approach  to  organizing  instruction  Is  slTnilar  In 
many  respects  to  looking  at  a  picture  through  b  zoom  lens 
on  a  movie  camera^ 

A  person  starts  with  a  wide-atiale  view,  which  alloV7a 
one  to  see  the  major  parts  of  the  piature  and  the  majo^ 
relationships  among  those  parts  <e,g*,  the  composition 
or  balance  of  the  picture)^  but  without  any  detail* 

The  person  then  gooms  in  on  a  part  of  the  picture, 
Aasinne  that^  Instead  of  being  continuous,  the  aoom  operates 
in  steps  or  discrete  levels ^     Zooming  in  one  level  on  e 
given  part  of  the  picture  allows  the  perEon  to  see  the  major 
subparus  of  that  part  and  the  inajor  relationships  among  those 
subparts.    After  having  studied  those  subparts  and  their  tttt&c^ 
rslationshlps,  the  person  could  then  zoom  back  oiit  to  tbe 
wicle-angle  view  to  review  the  other  parts  of  the  whole  plttura 
and  to  review  the  context  of  this  part  within  the  whole  pto- 
tura^ 

The  person  continues  this  pattern  of  rooming  in  ona  level 
to  see  the  major  subparts  of  a  part  and  zooming  back  out  fox 
oonteKt  and  r^vieWi  until  the  ^7hole  picture  has  been  seen  at 
the  first  level  of  detail*    Thmthe  person  can  follox^  the 
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same  20om-in/2oom-out  patterii  for  the  second  level  of  detail^ 
the  third  level,  and  so  oti^  \jntll  the  desired  level  of 
detail  is  reached^ 

In  a  similar  way  the  Elaboration  Theory  of  Instruction 
gtarts  the  student  with  en  overvlaw  of  the  major  parts  of 
the  suhject  tnatter,  it  elaborates  on  one  of  those  parts  to 
a  certain  level  o£  detail  (the  first  level  of  elaboration) » 
It  reviews  the  overview  and  mhovm  the  context  of  that  part 
within  the  overview  (an  expanded  overview) ,  it  continues 
this  pattern  of  elaboration/ eKpanded  overview  for  each  part 
of  the  overview  until  all  parts  have  been  elaborated  one 
level,  and  it  follows  the  same  pattern  for  further  levels  of 
elaboration.     Of  eourse,  it  must  he  remeinbered  that  the 
Eoom-iens  analDgy  is  just  an  analogy  and  therefore  that  it 
has  non-analogous  aspects.    One  such  dissimilarity  Is  that 
all  the  detail  of  the  picture  is  actually  present  (although 
usually  not  noticed)  in  the  wide-angle  view,  whereas  the 
detail  is  not  there  at  all  in  the  overview  of  the  subject 
inatterp  ,  - 

The  general-tO'-detaiied  organization  prescribed  by  the 
Elaboration  Theory  helps  to  Insure  that  the  learner  is  always 
aware  of  the  context  and  importance  of  the  different  topics 
that  are  being  taught.     It  allows  the  learner  to  learn  at 
the  level  of  detail  that  is  most  appropriate  and  tneaningful 
to  him  or  her  at  any  given  state  in  the  development  of  one's 
knowledge.    And  the  learner  never  has  to  struggle  through 
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a  series  of  learning  prerequisites  that  are  on  too  deep 
a  level  of  detail  to  be  interesting  or  meaningful  at  the 
Initial  stages  of  instruction.     Very  few  learning  pre^ 
requisites  (if  any)  exist  at  the  level  of  the  overview. 
As  a  learner  works  one*s  way  to  deeper  levels  of  detail^ 
increasingly  complex  prerequisites  will  need  to  be  intro- 
duced.    But  if  they  are  only  introduced  at  the  level  of 
detail  at  which  they  are  necessary,  then  there  will  only  he 
a  few  prerequisites  at  each  leveli  and  the  learner  will  want 
to  learn  those  prerequisites  because  he  or  she  will  see  their 
importance' for  learning  at  the  level  of  detail  that  now 
interests  him  or  her. 

Unfortunately^  the  zoom  lens  approach  has  not  been  used 
much  in  instruction^  in  spite  of  its  fundamental  siBipllcity 
and  intuitive  rationale^    Many  textbooks  begin  with  the  ^'lens" 
roomed  in  to  the  level  of  detail  deemed  appropriate  for  the 
intended  student  population,  and  they  proceed--with  the  *-leins-* 
locked  on  that  level  of  detall---to  pan  across  the  entire  sub-- 
ject  matter*    This  has  had  unfortunate  consequences  for 
sjmthesis,  retention,  and  motivation^    Many  instructional 
developers  begin  with  the  lens  goomed  all  the  way  in  and 
proceed  in  a  highly  fraginented  manner  to  pan  across  a  small 
part  and  zoom  out  a  bit  on  that  part,  pan  across  another  small 
part  and  zoom  out  a  bit  on  it^  and  so  on  until  the  tdiole 
scene  has  been  covered  and  to  some  limited  degree  integrated. 
This  has  also  had  unfortunate  consequences  for  synthesis i 
retention,  and  motivation.    And  some  educators  have  intu- 

iiu  107 


itively  groped  for  an  elaboration- type  approach  with  no  guide- 
liTies  on  how  to  do  it.     This  has  resulted  in  a  good  deal  less 
effectiveness  than  Is  possible  for  TnaximiEing  synthesis, 
retention^  and  motivation. 

The  Tnajor  reason  for  the  lack  of  utilization  of  the 
soom  lens  approach  in  instruction  is  probably  that  the  hler^ 
archtcal  approach  was  well-'artlculated  and  was  a  natural 
outgrox^th  of  a  strong  behavioral  orientation  In  educational 
psychology.     This  in  effect  put  "blinders"  on  most  of  the  fex^ 
people  who  were  working  on  instructional  design  strategies 
and  methodology* 

THE  ELABORATION  THEORY 

The  elaboration  theory  of  instruction  states  that  if 
cognitive  instruction  is  organised  In  a  certain  specified  way^ 
then  that  instruction  will  result  in  higher  levels  of  learn- 
ings  synthesis I  retention,  and  affect,  all  other  things  being 
equal*    There  Is  one  limitation  to  this  theory:  the  amaller 
the  atnount  of  Interrelated  subjsct^niatter  content,  tha  less 
difference  it  will  make.    With  a  small  enough    number  of 
toplas,  it  doesn^t  make  any  difference  how  you  sequeiice  them, 
whether  you  synthesize  them,  or  whether  you  summariEe  them 
(as  long  as  there  are  no  learning  prerequisite  relationships 
among  them)  *     The  following  is  a  description  of  that  '*aertain 
specified  way"  of  organlEing  instruction,  which  is  called  the 
Elaboration  Model  of  Instruc^tion. 

The  Elaboration  Model  of  Instruction  starts  by  present- 
ing knowledge  at  a  very  general  or  simplified  level-^-ln  the  ^ 
form  of  a  special  kind  of  overview.    Then  it  proceeds  Co  add 
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detail  or  complexity  In ^ "layers-'  across  the  entire  breadth 
of  the  content  of  the  course  (or  curriciiiiim)  >  one  layer  at 
a  time,  until  the  desired  level  of  detail  or  coniplexity  is* 
reached.     It  is  important  to  emphasize  that  the  Elaboration 
Model  presci'lbes  a  special  kind  of  overview,  and  it  pre-^ 
scribes  a  special  %my  in  which  the  elaboration  is  to  occur » 
The  Epitoine 

We  do  not  like  to  use  the  word  "overview"  because  its 
meaning  is  very  vague--it  means  different  things  to  different 
people.    Also,  we  believe  that  a  certain  specific  kind  of 
overview  is  superior  to  other  kinds ^    Among  other  things, 
our  overview  must  epitOTnize  the  subject  matter  that  is  to 
be  taught^ rather  than  summarizing  tt^    Hence,  we  have 
named  it  the  epitome  (e^plt  o '^me)  *    An  epitome  has  two 
"critical  characteristics"  i^^hich  distinguish  it  from  other 
types  Df  overviews s   (1)  it  epltoTnlzes  the  subject  matter  of 
the  course  (or  curriculum)  rather  than  summarizing  it^  and 
(2)  it  has  a  single  "prlentatlon"--lt  emphasizes  a  single 
type  of  contents  * 

With  respect  to  epitomizlng^  the  subject  matter  of  the 
course  (or  curriculum),  an  epitome  is  formed  by  "boiling  do^" 
the  course  content  to  Its  essence.     It  does  not  preview  all 
of  the  course  content;  rather  it  presents  a  few  fundamental 
topics  that  convey  the  essence  of  the  entire  content.  Those 
topics  are  chosen  or  derived  in  such  a  way  that  all  the 
remaining  course  content  provides  tnore  detail  or  more  complex 
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knowledge  about  the  epitoniG.    Although  an  epltOTne  is  very 
geiieral,  it  is  not  purely  abstract.     Since  ''genaral"  and 
"abstract"  are  often  confused,  this  distinction  mil  be 
discussed  in  greater  detail  shortly. 

With  respect  to  having  an  prientatlon,   the  epitome 
emphasizes  any  one  of  three  types  of  content:  concepts, 
procadureSi  or  principles^    A  concept  is  a  set  of  objects ^ 
events,  or  ideas  that  have  certain  characteristics  in  coimon. 
Knowing  a  concept  entails  being  able  to  identify^  recognise, 
classify,  or  describe  what  somBthing  Is,    A  procedure  la  a 
set  of  actions  that  are  intended  to  achieve  an  end.    It  is 
often  referred  to  as  a  skill ,  a  technique,  or  a  inethod. 
Knowing  a  procedure  entails  knowing  how  to  do  something. 
A  pf Inciple  is  a  change  relatlonshlp--it  indicates  the  re^ 
lationship  bet^jeen  a  change  in  one  thing  and  a  change  in 
something  else.     It  describes  causes  or  affects  by  identify- 
ing what  will  happen  as  a  result  of  a  given  change  (the  ef- 
fecc)  or  v7hy  something  happens  (the  cause).    These  three 
different  emphases  are  referred  to  as  a  conceptual  orianta-- 
tion^  a  procedural  orlentatlpni  and  a  theoretical  orientaeion^ 
rBSpectlvelyi  and  the  orientation  is  selected  on  the  basis 
of  the  general  goals  or  purpose  of  the  course  (or  curriculijm)  , 
All  three  types  of  content  may  appear  in  the  epitome,  but  one 
type  receives  primary  emphasis  j  and  the  epitome  is  forwed 
by  epitoralEing  the  orientation  content  and  then  introducing 
vhatever  of  the  other  two  types  of  content  are  highly  relevant 
More  will  be  said  about  this  below* 
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We  mentioned  above  that  an  epitome  is  very  general  but 
is  not  purely  abstract*     The  terms  ''general'*  and  ''abstract**  are 
often  confused^     It  is  helpful  to  tblnk  of  three  continuai 
(1)  general  to  detailed^   (2)  simple  to  cotnplex,  and  (3)  abstract 
to  concrete^     The  first  two  are  very  similar  to  each  other, 
but  the  third  is  very  different  (see  Figure  2)^ 

Insert  Figure  2  about  here 

The  general"- to-detailed  continuum  refers  primarily  to  a..cpn-* 
tinuum  formed  by  subdividing  things  (concepts  or  procedures) 
or  by  lumping  things  (concepts  or  procedures)  together^ 
"General'-  has  breadth  (things  lumped  together),  while  "detailed" 
is  usually  narrow  (subdivisions).     In  Figure  2a'  "polar  bear" 
is  a  more  detailed  concept  than  -'animal,'*    The  simple-- to-complex 
continuum  refers  primarily  to  a  continuum  fornied  by  adding  or 
removing  things  (principles  or  procedures)*    "Simple"  has  few 
things^  while  "complex"  has  many  things^    In  figure  2b,  the  pro- 
cedure for  subtracclng  multi-digit: numbers  la  more  complex  than 
the  procedure  for  subtracting  single-digit  numbers.  Additional 
ccmpleKity  can.ba  added  by  introducing  sub^procedures  for  "bor- 
rowing" when    the  top  number  is  smaller  than  the  bottom  number^ 
The  ab B t r a c t - 1 o - c oti c r e t e  continuum  refers  to  tangibility,  and 
there  are  two  major  types  of  tangibility.    First,  generalities 
are  abstractpand  instances  are  usually  concrete--- the  definition 
of  a  tree  Is  abstract,  while  a  specific  tree  (an  object)  is  con- 
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crete.     This  is  the  moat  IniporLani  abBLraaL-to^concrete 
continuum  for  instructional  theory,     Secondp  some  concepts 
are  considered  abstract  because  their  instances  are  not 
tangible,     "Intelligence-'  is  a  good  eKample  of  an  abstract 
concept.     This  second  abstract- to-concrete  continuum  is 
largely  irrelevant  for  our  purposes* 

On  the  basis  of  these  distinctions,  an  epitome  is 
always  either  very  general  or  very  simple--it  must  be,  in 
order  to  epitomize  the  instructional  content.     But  It  should 
never  be  purely  abstract.    According  to  Merrill's  Component 
Display  Theory,  It  should  contain  the  following  for  each 
topic  it  presents:  a  generality  (e.g*,   the  definition  of  a 
concept),  some  instances  of  that  generality  (e*g*|  examples 
of  the  concept),  and  some  practice  for  the  student  In  applying 
the  generality  to  new  instances t    An  epitome  usually  contains 
about  six  (plus  or  minus  three)  topics-- that  is,  about  six 
different  generalities ^  along  mth  their  instances  and  prac- 
tice items*     These  topics  may  be  any  combination  of  concepts, 
procedures,  and/ or  principles. 

Figure  3  illustrates  the  nature  of  a  theoretical  epitome 
and  of  a  conceptual  epitome  for  an  introductory  conrse  in 
ecoTnonics,  and  Figure  4  illustrates  the  nature  of  a  procedural 
epitome  for  a  course  in  literature. 


Insert  Figures  3  and  4  about  here 


1,  The  law  (principle)  of  supply  and  datnatid* 

a,  The  principlo  of  what  causes  changes  to  occur  in  the  quant- 
ity demanded  and  the  quantity  supplied  (prlcG  changds) , 

b.  The  principle  of  why  prices  change  In  a  free  market 
econoiny  ^ 

2,  The  principle  of  why  changes  occur  in  supply  schedules  or 
detaand  schGdulcs . 

3,  The  concepts  of  supply^  supply  schedule,  and  supply  eurve^ 
4*  The  concepts  of  demand,  demand  schedule,  and  demand  curve. 
5^     The  concept  of  changes  in  quantity  supplied  or  deraandQd. 

6*     The  concept  of  changes  in  supply  schedules  or  deniand  schedules, 
7»     The  concept  of  equilibrium  price* 

Practically  all  principles  of  economics  can  be  viewed  as  elabor- 
ations on  the  law  of  supply  and  demand,  including  those  that 
relate  to  monopoly,  regulation,  price  fixing,  and  planned  economies. 


Figure  3a*     The  instructional  content  for  a  theoretical 
epitome  for  an  introductory  course  in  economics. 


1.  Definition  of  economics 

2,  Definitions  of  subdivisions  of  economics: 

a.  Definition  of  macro  economics 

b.  Definition  of  micro  economics 

c.  Definition  of  comparative  economics 
d*    Definition  of  international  economics 
e*    Definition  of  labor  economics 

f .    Definition  of  rimnagerial  economics. 

Practically  all  concepts  in  economics  can  be  viewed  as  elabora- 
tions on  these  concepts  (l;e*,  as  further  subdivisions--either 
parts  or  kinds'^^'of  these  concepts)  . 


Figure  3b.  ^The  Instructional  content  for  a  conceptual 
epitome  for  an  introductory  course  in  economics. 
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1.     There  are  three  major  stages  In  the  multidimensional 
analysis  and  iiiterpretation  of  creative  literature: 

Identifying  the  elenients  of  the  dramatic  frameworks- 
character  and  plot. 

b.  Combining  the  elements  into  composites  appropriata 
for  analysis  of  their  literal  nieaning--analysis  of 
character  in  tertns  of  plot. 

Figuratively  interpreting  the  eleTnents--aymbolism 
through  character,  Tiiood,  tonep 

d.     Making  a  judgTnent  of  worth-'-personal  rele-vance, 
univer  s  all ty . 

(This  procedure  is  simplified  by  introducing  only  two  elements 
for  the  analyses  in  a  and  h,  three  in  c,  and  two  in  d*  It 
is  further  simplified  by  introducing  only  those  procedures 
and  concepts  necessary  for  the  analysis  and  interpretation  of 
a  short  poem*     Complexity  is  later  added  by  increasing  the 
number  of  elements  used  in  each  stage  of  analysis  or  inter-^ 
pretation  and  by  introducing  procedures  and  concepts  needed 
for  analyzing  and  interpreting  more  cornpllcated  types  of 
creative  literature*) 

2*     Concepts  necessary  for  performing  the  procedure  in  1. 


a. 

character 

b. 

plot 

c. 

symbolism 

d. 

mood 

tone 

f . 

univerEallty 

Figure  4.  The  instructional  content  for  a  procedural  epitome 
for  an  Introductory  course  in  literature. 


A  Level-l  Elaboratlori 

A  level-1  elaboration  is  a  part  of  the  instruction  that 
provides  some  inore  detailed  or  compleK  knowledge  on  a  topic 
(or  set  of  topics)  that  was  introduced  to  the  student  in  the 
epitotiie^     It  should  not  Include  all  of  the  more  detailed  or 
compleK  knowledge  on  that  topic ^     Rather^  a  level-1  elaboration 
should    be  an  epitome  of  all  of  the  more  detailed  or  complex 
knowledge  on  that  topic ^  just  as  zooming  in  one  level  provides 
a  slightly  more  detailpd .  wide-angle  view  of  one  part  of  the 
whole  picture*     There  may  be  as  many  level-1  elaborations  as 
there  are  topics  in  the  epitome^  but  there  does  not  have  to 
be  a  one-to-^one  correspondence.     It  is  possible  that  each  of 
the  topics  in  a  level^l  elaboration  may  elaborate  to  some 
extent  on  all  of  the  topics  in  the  epitome  or  perhaps  even 
on  a  relationship  among  those  topics. 

The  depth  to  which  a  level-1  elaboration  should  elaborate 
on  a  part  of  the  epitome  la  somewhat  variable  (l^e,,  the  dls-- 
Crete  levels  on  the  zoom  lens  are  variable,  not  always  constant 
and  equal  In  the  amount  of  detail  added)  ,     The  most  Important 
factor  for  deciding  on  the  depth  of  a  given  level-1  elaboration 
Is  student  learning  load.     It  Is  important  that  the  student 
learning  load  be  neither  too  large  nor  too  small ^  for  either 
will  impede  the  Instruction's  efficiency,  effectiveness 
(especially  for  retention),  and  appeal.     The  number  of  topics 
that  represent  the  optimal  student  learning  load  will  vary  with 


such  factors  as  student  ability,  the  compleKlty  of  the  subject- 
inatter  topics^  and  student  prefainiliarity  with  the  topics. 
The  breadth  of  a  level- 1  elaboration  will  usually  be  fairly 
difficult  to  adjustp     Hence  optltnizing  the  student  learning 
load  In  a  given  elaboration  can  often  be  done  only  by  varying 
the  dep th  of  that  elaboration. 

Figure  5  illustrates  the  nature  of  a  level-1  elaboration 
on  the  theoretical  epitome  In  Figure  3,  and  Figure  6  Illus- 
trates the  nature  of  a  level-1  elaboration  on  the  procedural 
epitome  in  Figure  4. 

Insert  Figures  5  and  6  about  hera 

Other  Elaborations 

A  level-2  elaboration  is  Identical  to  a  level-1  elaboration 
except  that  It  elaborates  on  a  topic  (or  set  of  topics) 
introduced  in  a  leval-1  elaboration  rather  than  in  the  epitome. 
In  a  similar  manner,  a  level -3  elaboration  provides  more  detail 
or  compleKity  on  a  topic  (or  set  ot  topics)  introduced  in  a 
level-2  elaboration,  and  so  on  for  elaborations  at  deeper  levels 
of  detail/complexity^    In  all  caseSi  an  elaboration  at  one 
level  of  detail/ complexity  should  be  an  epitome  for  all  the 
elaborations  that  elaborate  on  it. 

According  to  this  kind  of  organization,  elaborations 
that  are  on  the  same  level  are  very  different  from  each  other 
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Ip  Principle  of  increasing  marginal  costs  as  an  explanation  for  the 
shape  of  the  supply  curve. 

2»  Princtple  of  profit  maKimization  for  individual  firms. 

3*  Procedure  of  marginal  analysis  to  arrive  at  profit  Tnaximigation* 

4,  Concepts  of  flKed  and  variable  costs , 

5p  Concepts  of  total,  average^  and  marginal  costs. 

6,  Concepts  of  braak-even  point  and  shut-down  point* 


Figure  5*     The  instructional  content  for  a  level-1  elaboration 
on  the  theoretical  epitOTTie  in  Figure  3a,     It  elaborates  on  the 
supply  aspect  of  the  law  of  eupply  and  demand  by  presenting 
more  complex  principles  that  relate  to  supply. 
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1.  How  to  identify  other  elements  of  the  dramatic,  frarne- 
i^ork-" set ting,  perspective,  and  language, 

2.  How  to  combine  the  alements  into  coTnposites  appropriate 
for  analysis  of  their  literal  meaning-- (1)  analysis  of 
character,  plot,  and  setting,   (2)  analysis  of  persepctive, 
character,  and  plot,  and  (3)  analysis  of  language, 

3.  Concepts  of  setting,  perspective,  and  language, 

4.  Concepts  of  types  and  patterns  of  iinagery  (In  language), 

5.  Procedure  for  analyzing  Imagery, 

6.  Concept  of  prosody, 

7.  Procedure  for  analyzing  prosody. 


Figure  6,     The  instructional  content  for  a  level-1  elabora-- 
tion  on  the  procedural  epitome  in  Figure  4*     It  elaborates 
on  stages  a  and  b  (which  must  be  elaborated  at  the  same  time 
because  of  their  interrelatedness)  by  adding  elements  that 
need  to  be  identified  (In  stage  a)  and  analyzed  In  comblTiatlon 
(in  stage  b) * 
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vJith  respect  to  the  instructional  content  they  contain  (i»e», 
their  topics  are  very  different  from  each  other)'  but  elabor- 
ations that  are  on  different  levels  are  very  similar  to  each 
other  with  respect  to  their  instructional  content  (i*e. ,  their 
topics  are  very  sitnllar)  because  each  level  has  the  same 
content  as  the  previous  levels,  only  at  a  level  of  greater 
detail/compleKlty s 
Expanded  Epitome 

After  each  elaboration,  the  instruction  presents  a  stam- 
marlzer  and  an  eKpanded  epitome,  equivalent  to  the  Eoom-out-' 
for--context-and-review  activity  In  the  200m  lens  analogy.  The 
summarigar  is  comprised  of  a  concise  generality  for  each  topic 
presented  in  the  elaboration.     The  expanded  epitome  (a)  syn- 
thesizes the  topics  presented  within  the  elaboration  Cinternal 
synthesis)  and  (b)  shows  the  relationship  of  those  topics  (and 
relationships)  to  the  rest  of  the  topics  (and  relationships) 
that  have  been  taught  (eKternal  synthesis) . 
Summary  of  the  Elaboration  Model 

In  surEmary^  the  ElabDratlon  Model  is  as  follows  (see  Figure 
7)*     First,  the  epitome  is  presented  to  the  student.    Then  a 
level-1  elaboration  is  presented  to  provide  more  detail  on  an 
aspect  of  the  orientation  content  in  the  epitome  (that  aspect 
which  is  most  important  or  contributes  most  to  an  understanding 
of  the  whole  orientatton  structure)  .    Next  a  surmtiarlger  and  an 
eKpanded  epitome  are  presented.    Another  level-1  elaboratlQn 
and  its  summariEer  and  expanded  epitome  are  presented.  This 
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pattern  of  level- 1  elaboration  follo%^;ed  by  its  suminariEer  and 
expanded  epitome  continues  until  all  aspects  of  the  orifintation 
content  that  %^ere  presented  in  the  epitome  have  been  elaborated 
one  level.     Then  a  level-- 2  elaboration  Is  presented  to  provide 
mora  detail  on  an  aspect  of  the  orlentatton  content  that  was 
presented  in  one  of  the  level-1  elaboratlortS  *    As  always,  this 
elaboration  is  followed  by  a  summariger  and  an  expanded  epitoine,. 
This  pattern  continues  until  all  of  the  aspects  of  the  orienta- 
tion content  presented  in  all  of  the  level-1     elaborations  have 
been  elaborated  one  level  (unless  the  objectives  of  the  course 
or  the  nature  of  the  subject  matter  eKempt  a  level-1  elabora- 
tion from  being  further  elaborated) ,    Additional  levels  of  elab- 
oration are  provided  in  the  same  manner-- an  elaboration  followed 
by  a  sUTRmarlzer  and  an  eKpanded  epttome--^xtntll  the  level  of 
detail/complexity  specified  by  the  objectives  is  attained  in 
all  aspects  of  the  orientation  structure  (and  supporting 
structures)  of  the  course » 

Insert  Figure  7  about  here 

USING  THE  ELABORATION  MODEL 
We  have  developed  a  fairly  detatled  set  of  procedures  for 
designing  instruction  according  to  the  Elaboration  Model 
(Reigoluth,  et  al,  1978)^    A  major  part  of  those  procedures  is 
analyzing  the  instructional  content  as  to  four  different  types 
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EPITOME 


(1)  EPITOME 


PRIMARV^LEVEL 
ELABORATIONS 


1^ 


(2)     A  PRIMARY-LEVEL 
ELABORATION 
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(2)    ANOTHER  PRIMARY- 
UVEL  ELABORATION 


SUMMARIZiR  AND 
EXPANDED  EPiTOME  ON 
THAT  ELABORATION 


SUMMARI2ER  AND 
(3)  EXPANDED  EPITOME  ON 
THAT  ELABORATION 


SECONDARV-LEVEL 
ELABORATIONS 


{4)  ASeCONDARY-LlVEL 
ELABORATION 


SUMMARIZER  AND 
^    (5)  EXPANDED  EPITOME  ON 
THAT  ELABORATION 


{4)   ANOTHER  SECONDARV» 
LeVEL ELABORATION 

SUMMARIZER  AND 
!5)  EKPANDED  EPITOME  ON 
THAT  ELABORATION 

LOWER-LEVEL 
ELABORATIONS 


(6)    SO  ON  FOR  TERTIARY  AND  FOURTH- 
LEVEL  ELABORATIONS,  IF  NEEDED 


I 


(7)   TERIViiNAL  SUMMARIZER 
AND  TERMINAL  EPITOMB 


Figure  7.    A  diagramatic  representation  of  the 
Elaboration  Model  of  Instructlori. 
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of  subject-matter  structures ^  concaptual,  procedural,  theoretical, 
and  learning,     (Learning  structures  shoi-j  learning  prerequisite 
relations  withirf  the  subject  matter. )     It  is  bayond  the  scope 
of  this  paper  to  describe  and  Illustrate  each  of  these  four 
types  of  structures.     The  interested  reader  is  referred  to 
Reigeluth,  Merrill,  6z  Bunder son,  1978 • 

There  are  six  major  steps  for  destgniTig  Instruction  accord- 
ing to  the  Elaboration  Model  (see  Figure  8),     First_,  one  must 
select  an  orientation--elther  conceptual^  procedural,  or 
theoretical--'On  the  basis  of  the  goals  or  purpose  o£  the  in-' 
truction.     Second,  one  must  develop  an  orientation  structure 
for  that  orientation.     It  depicts  the  orientation  content 
(either  concepts,  procedures,  or  principles)  in  the  most 
detailed/ complex  version  that  the  studeiit  needs  to  learn 
This  is  a  form  of  content  analysis  or  task  description.  Then 
the  orientation  structure  Is  analyaed  in  a  systematic  manner 
to  determine  which  aspect (s)  of  the  orientation  content  will 
be  presented  in  the  epitome  and  which  aspects  will  be  presented 
In  each  level  of  elaboration.     In  this  wy  the  "skeleton"  of 
the  instruction  is  developed  on  the  basis  of  epitomising  and 
elaborating  a  single  typ^    of  content p 

Insert  Figure  8  about  here 


The  fourth  major  step  is  to  embellish  the  "  skeleton"  by 
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2b.  Procedural 


Idontify  oil  useful 
steps  and  slternativs 
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and  eembine  them 
Into  a  procedural 
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Theofltieil 
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3a.  ConEeptual 
Prune  the  orlDntatlsn 
stfuctyrd  to  form  the 
fipitariie,  Add  bock 
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level  of  diaboration, 

%  Pfsecdgral 
Lump  steps  and 
bfafiches  together  to 
form  the  epitome. 
8rilk  ppart  eagh  pn^ 
irol  step  to  form  each 
level  of  elaboration, 

3e.  Theoretical 


oil  unattainad 
conlcxtual,  prQcodur- 
al,  and  eKplariatory 
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cofresponding  sup' 
porting  stfucturei, 
Then  Idontify  all 
unattalned  learning 
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make  their  eorres- 
ponding  learnlfio 
stnictufei, 


Divide  caeh  level  of 
eloboration  Into  lis 
Individual  elaboro' 
tions,  according  to  the 
part  of  the  epitome, 
or  of  an  ilsboration, 
that  is  being 
elabaratod, 


Design  the  epitome. 


9  general  i 
0  constructi 
f  symmarizer 
9  epitonie 
(synthesiier) 

Design  each  elabora' 
tion  in  the  priraT 
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e  sumniariier 
i  expinded  epitomi 
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epitome, 


I  ofientation 
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ie  orientatisn 
stfueturi  to  ktm  the 
epitonni.  Uio  rank' 
Ofder  of  j^portanoo 
'  and/Df  a  parallel  mn* 

'      ceptyal  ilructure  to 
idpntify  prinaj|5leifQr 
Dochlovoldf 
yloboriitien, 

Figure  §  •  The  iIx*stop  doiip  proeodurc  for  structuring  the  instruction  in  any  course  entailing  cognitive  mmzL 


adding  the  ocher  two  types  of  conterit  at  the  lowest  appropriate 
levels  of  details     This  is  usually  done  by  "nesting"-  the 
remaining  subject -matter  structures  within  different  parts  of 
the  skeleton.     (This  Tnay  include  some  Isolated  structures  or 
topics  in  the  orientation  content  that  did  not  fit  in  with  the 
orientatton  structura  in  f^-rmtng  the  skeleton  ^  Learning 
prBrequisites  are  one  of  the  considerations  that  enter  in 
at  this  point* 

Having  allocated  all  of  the  instructional  content  to  the 
different  levels  of  elaboration^  it  is  now  important  to  es-- 
tablish  the  scope  and  depth  of  each  Individual  elaboration 
that  will  comprise  each  level.     The  scope  is  usually  pre- 
deterniined  by  the  orientation  topic  and  its  necessary  support- 
ing topics^  although  two  orientation  topics  can  be  lumped 
together  into  a  single  elaboration^ and  it  would  be  possible 
(though  not  advisable)  to  add  extra  supporting  topics*  The 
depth  is  then  determined  on  the  basis  of  achieving  an  optiinal 
student  learning  load,  as  described  above* 

SiKth  and  finally,  some  of  the  internal  structure  of 
each  elaboration  within  each  level  can  be  planned*     The  ' 
sequence  of  topics  within  an  elaboration  is  decided  on  the  basis 
on  contribution  to  an  understanding  of  the  whole  orientation 
structure  (but  within  the  constraints  of  learning  prerequisites) , 
and  the  locations  of  synthesizers  and  sunmariaers  are  i^p.so 
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determined. 

This  concludes  the  *'macro"  design  process,  at  x/hich 
point  the  -'micro"  design  process  begins---decisions  as  to 
how  to  organize  the  Instruction  on  a  single  topic. 

THE  NEED  FOR  RESEARCH 

The  model  and  prQcedures  as  described  above  have  under-^ 
gone  very  limited  f ield-^testlng  and  virtually  no  research. 
It  may  turn  out  that  having  a  complete  expanded  epitome  after 
every  single  elaboration  is  Inefficient  and  unnecessary  (es-- 
peclally  after  lower-level  elaborations),.  It  may  also  turn 
out  that  it  is  unnecessary  for  a  student  to  study  all  level-1 
elaborations  before  proceeding  to  a  level-2  elaboration.  This 
^70uld  have  Important  implications  for  learner-controlled 
selection  and  sequencing  of  topics--a  student  could  nox^  truly 
follow  one's  interests  in  approaching  a  subject  matter.  This 
vould  be  particularly  valuable  in  adult  and  continuing  education 
contexts. 

It  is  also  likely  that  a  large,  full-^scale  field  test  of 
the  design  procedures  will  reveal  more  effective  and  efficient 
ways  to  design  instruction  according  to  the  models 

The  Elaboration  Model  as  developed  to  date  is  a  tentative 
move  in  a  much-needed  direction*     It  does  not  yet  have  the 
maturity  and  validation  of  the  currently  used  approaches  to 
Instructional  design,  but  the  need  for  alternatives  should  be 
clear.    And  there  is  great  potential  for  the  Elaboration  Model 
to  meet  that  need. 


J       It  is  becoming  increasingly  evident  that  there  are  many 
deficiencies  in  the  hierarchical  task  analysis  approach  to 
organising  instruction  on  the  macro  level*    The  resulting 
instructional  designs  are  usually  fragmented,  demotivating, 
inaonsistent  i^ith  learning  theory,  and  at  best  a  very  In- 
complete basis  for  organizing  instruction.    The  Elaboration 
Theory  of  Instruction  was  developed  as  an  alternative  that 
overcomes  these  deficiencies*     But  it  is  emphasized  that  it 
is  but  a  partial  theory  of  lnstructlon--lt  only  deals  with 
macro  strategies  for  organizing  instruction  (see  Figure  1 
above) , 

The  Eoom-lens  analogy  was  presented  as  an  introduction 
to  the  nature  of  the  Eiaboration  Theory  of  Instruction,  A 
person  starts  with  a  wide-angle  view  and  then  proceeds  to 
^oom  in  one  level  for  detail  on  a  part  of  the  picture  and  200m 
out  for  review  and  context,'  After  the  whole  first  level  has  .be 
studied,  the  same  200m- in/ zoom-out  pattern  is  followed  for 
the  second  level  of  detail,  and  so  on  until  the  whole  picture 
has  been  studied  at  the  desired  level  of  detail.  Alternative 
approaches  to  designing  instruction  were  contrasted  in  terms 
of  this  analogy,  and  their  deficiencies  were  mentioned. 

Next,  the  Elaboration  Model  of  Instruction  was  described. 
It  starts  by  presenting  a  special  kind  of  overview,  called 
the  "epitome",  which  (1)  epitomizes  the  instructional  content 
rather  than  suimiariElng  It  and  (2)  has  a  single  "orientation" 
(a  single  type  of  content)  .     The  single  orientation  may  be 


either  conceptual,  procedural,  or  theoretical.    And  to  epito^ 
mize  the  content,  the  epitome  contains  one  or  two  very  general/ 
simple  topics  from  the  orientation  content;  but  the  epitome 
should  not  be  purely  abstract---' the  epitonie  should  be  formed 
in  such  a  way  that  concrete  instances  and  practice  can  be 
presented  on  each  topic ^    Some  of  the  other  two  types  of 
content  may  also  be  included  in  the  epitome  if  they  are  highly 
relevant  to  the  orientation  content  topic(s).» 

After  presenting  the  epitome ,  the  model  calls  for  adding 
detail  or  compleKity  in  "layers"  across  the  entire  breadth 
of  the  content,  one  layer  at  a  tlmei  until  the  desired  level 
of  detail/ compleKity  is  reached*    First ,  level-1  elaborations 
are  presented,  each  of  which  provides  more  detail/complexity 
on  one  aspect  of  the  epitome.    After  all  aspects  of  the 
epitome  have  been  elaborated  in  this  manner,  level-2  elabora- 
tions are  presented*    Each  of  them  elaborates  on  one  aspect 
of  a  ievei^l  elaboration.    Additional  levels  of  elaborations 
are  added  as  necessary  to  reach  the  level  of  detail/compleKity 
called  for  by  the  objectives*     The  amount  of  content  in  each 
elaboration  must  be  carefully  planned  so  as  to  represent  an 
optimal  student  learning  load*    Also,  each  elaboration,  regard*- 
leas  of  levels  is  followed  by  a  aunmiarlzer  and  an  expanded 
epitome* 

Next  the  procedure  for  using  the  elaboration  model,  in  the 
design  of  new  instruction  was  summarized*  The  six  major  steps 
are  shotim  in  Figure  8  above. 
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Finally,  the  need  for  continued  research,  field-testing, 
d  development  of  the  Elaboration  Model  %-7as  emphasized. 
The  Elaboration  Model  as  developed  to  date  is  a  tentative 
move  m  a  much=needed  direction.     The  need  for  alternatives 
to  the  hierarchical  approach  should  be  clear,  and  there  Is 
great  potential  for  the  Elaboration  Model  to  meet  that 
need. 
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FOOTNOTE 

subject  matter  structure  is  something  which  shows  a 
single  kind  of  relationship  that  exists  within  a  subject 
matter.    Figure  2a  shows  part  of  a  subject-inatter  structure. 
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A  Consumer's  Guide  to  Common  Flaws  In  Research 


Dennis  M,  Roberts 
The  Pennsylvania  State  University 


*  The  present  paper  is  a  revised  and  condensed  version 
of  a  paper  "Common  Flaws  In  Research  Design"  by 
William  Rabinowitz  and  Dennis  Roberts  published 
in  the  NABTE  Journal,  1977, 
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Planning  the  Research  Study 

The  purpose  of  research  is  to  shed  light  on  important  questions  phrased 
In  testable  forms.     In  general,  one  seeks  to  establish  functional  or  causal 
relationships  between  independent  and  dependent  variables*  However, 
designing  studies  to  establish  these  relationships  or  causes  involves  many 
potential  hazards.     In  any  case  though^  the  investigator  has  an  obligation 
to  plan  the  study  in  order  to  maximize  the  interpretability  of  the  findings. 
Unfortunately,  there  are  many  basic  flaws  that  can  seriously  hinder  results 
interpretation.    The  present  brief  paper  is  a  sequential  walk  through  a 
simple  research  setting—  a  two  group  eKperiment^-=  to  indicate  common  (but 
avoidable)  design  difficulties  that  will  jeopardize  the  usefulness  of  the 
obtained  data. 
Common  Flaws 

Figure  1  presents  a  flow  chart  type  diagram  of  sevfciral  important  steps 
In  the  research  process.     Obviously ^  it  is  kept  simple--  many  details  of  actual 
design  execution  have  been  omitted  because  of  space  limitations.     Assume,  for 


Insert  Figure  1  about  here 


purposes  of  the  remaining  discussion,   that  "the"  important  question  is 
"Will  the  use  of  electronic  calculators  improve  elementary  school  students' 
attitudes  towards  mathematics?" 

^ •     Failure  to  select  subjects  from  the  population  to  which  generalizations  are 
to  b^e  made.  -    -  — 

This  item  deals  specifically  with  what  Campbell  and  Stanley  (1963)  refer 
Co  as  external  validity.     Clearly,  the  question  implies  that  an  answer  is 
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Figurf  1.  Steps  and  posiiblf  deiign  fliwi  m  an  eKpffiment  to  study  the  fffcct  o(  frtatment  variable  A  on 
variable  B, 
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desired  for  the  overall  population  of  elenientary  school  students.    Therefore , 

there  should  be  evidence  in  the  methods  section  that  shows  students  at 

varying  elementary  grade  levels  are  included  in  the  sample.     Secondly,  there 

must  be  evidence  that  the  students  from  these  several  grade  levels  were 

drawn  (for  the  study)  in  a  way  to  assure  a  representative  sample*     If  the 

sample  differs  In  important  ways  from  the  populations  the  results  may  not 

generalize  to  the  population  of  Interest  as  stated  in  the  reeearch  question* 

2*     Failure  to  use  randoniization^  In  jssignlng  _students  to  experimental  and 
control  groups. 

It  is  absolutely  essential  that  both  groups  be  equivalent  at  the  onset  of 
the  experlmenc.     Differences  between  the  groups  on  the  dependent  variable  chat 
appear  at  the  conclusion  of  the  study  will  Invariably  be  interpreted  as 
resulting  from  the  impact  of  the  independent  variable.    However ,  this  cQnclusion 
cannot  be  justified  if  groups  were  different  initially  in  important  (and 
unknown)  ways*     Unfortunately ^   there  is  no  way  to  absolutely  guarantee  equlva^ 
lence  of  both  groups  initially*    However^  random  assignment  does  ensure 
equivalence  within  the  limits  of  chance*     Since  random  fluctuations  (chance 
aberrations)  are  estima tables  random  assignment  offers  a  way  to  cqntrql^  the 
initial  makeup  of  the  two  groups.     If  the  method  section  does  not  give  specific 
indications  that  students  were  assigned  at  random  to  the  two  groups,  the  entire 
set  of  results  are  very  suspect. 

3.     Failure  to  use  an^dequa^e  numbe^r  of  students , 

If  the  hope  is  to  support  the  notion  that  use  of  calculators  will  irngrove 
attitudes  towards  mathematics j   then  we  certainly  want  to  reject  the  null 
hypothesis  (jj  Attitude  with  Calculators  *  p  Attitude  without  Calculators)  if_ 
our  notion  is  actually  correct .     In  this  sltuationj  rejecting  the  null  in  the 


study  when  our  position  is  the  true  state  of  affairs  is  a  correct  decision 
to  have  reached.     This  type  of  correct  decision  is  called  the  power  of  a 
statistical  test.    Unfortunately,  unless  your  independent  v^arlable  is  very 
potent j  power  is  greatly  influenced  by  sample  size*    Also,  unfortunately,  the 
ability  to  build  a  certain  degree  of  pother  into  a  study—  while  doable-^=^  is 
fairly  complicated,    A  paper  by  Hopkins  (1974)  may  be  helpful  to  the 
Interested  reader.     In  the  present  studyj  it  ii  doubtful  that  calculators 
are  going  to  work  miracles  on  students'  sttitudes  towards  mathematics.     Thus  ^ 
a  failure  to  use  a  "sufficient"  sized  sample  is  certainly  dooined  to  a  retention 
of  the  null  hypothesis* 

4^     Failure  to  maintain  control  oyer  manipulation  of  the  independent  variable. 

A  necessary  assumption  is  that  students  in  the  two  groups  werGj  in  fact, 
treated  differently  throughout  the  period  of  the  study.  In  our  contextj  this 
r^gLulres  that  no  control  students  have  access  to  calculators  during  the 


experiment*     While  this  may  be  controllable  during  the  math  classes j  control 
students  might  have  access  to  and  use  calculators  in  other  classes  (science?) 
and/or  outside  of  school*     Such  contamination  is  vary  likely  to  reduce  the 
differences  between  the  two  groups  on  the  dependent  variable.    What  will  look 
like  an  Ineffective  l^nde^endent_  variable  actually  disguises  the  possibility 
that  the  calculator  did  have  an  effect;  however ,   inadvertencly  in  groups 
5,     Failure  to  use  reliable  and/or  valid  measures  of  the  dependent  variable. 

The  original  question  posed  stated  that  the  important  depindent  variable 
of  interest  is  "attitudes  toward  mathematics".    Assuming  for  a  moment  that 
such  a  construct  exists,  then  at  issue  is  whether  the  construct  can  be 
measured  consistently.     Without  some  data  being  presented  to  demonstrate 
reliability  of  the  measure  In  this  particular  research  setttngj  then  it  is 
unlike ly  that  dlfferencGs  botween  tho  groups  on  the  dependent  measure  wllJ 
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occur.    Assurrilng  reasonable  rellabilityi  however,  a  more  important  issue  is 
whecher  the  dependent  measure  in  the  study  Is,  in  fact,  measuriiig  attitudes 
toward  mathematics.    Obviously,  any  comparison  between  the  groups  on  a 
measure  that  isn't  tapping  the  important  construct  of  incerest  is  useless  and 
certainly  n£  test  of  the  hypothesis  we  might  have  generated  ftom  our  research 
question  (i.e*,  calculators  improve  attitudes  towards  mathematics), 
6 .     Failure  to  use  an  appropriate  statistical  test. 

One  area  where  flaws  shouldn't  occur  but  do  quite  often  is  the  inappropriate 
use  of  certain  statistical  tests.     The  emphasis  here  is  not  on  using  a  test 
that  is  less  powerful  than  others  but  rather  on  using  a  test  incorrectly.  A 
common  example  here  Is  using  analysis  of  covariance  (ANCOVA)   to  make  statistical 
adjustments  for  the  fact  that  groups  were  not  equivalent  initially.     What  this 
means  (in  all  probability)  is  that  a  statistical  technique  is  supposed  to 
correct  for  non^randomlssation  of  students  to  the  experimental  and  control  groups. 
Unfortunately  J  ANCOVA  can't  make  non=^  random  assignment  random;  no  statistical 
test  can  correct  for  that  difficulty,    ANCOVA  has  two  sets  of  assumptions  that 
need  to  be  met  in  order  to  be  Interpreted  properly—  there  are  those  for 
analysis  of  variance  (ANOVA)  and  regression.     If  random  assignment  has  been 
made  and  there  Is  a  reaBonable  correlation  between  a  covarlate  (measured-^ 
incidentally---  before  the  experiment  begins)  and  the  dependent  variable,  then 
ANCOVA  may  reduce  the  error  term,  hence*  giving  a  better  chance  to  reject  the 
null  hypothesis.     The  interested  reader  Is  referred  to  an  article  on  ANCOVA 
by  Kennedy  (1977)   for  elaborations  on  this  topic. 
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Closing  Comments 

The  present  focus  has  been  on  several  coimnon  flaws  that  occur  in  even 
the  simplest  research  design*     Obviously,  many  other  flaws  may  creep  into 
research  studies  such  that  the  output  data  are  rendered  auspect.  Firsts 
many  studies  do  not  use  an  appropriate  research  design  to  handle  the 
hypotheses  being  tested*     Second »  many  research  questions  are  fortiiulated  in 
such  a  way  that  no  data  will  be  able  to  support  the  hypntheses.     And  third, 
many  Investigations  are  conducted  without  proper  consideration  of  the 
relevant  variables  that  might  have  an  Influence  on  the  dependent  measures. 
All  these  additional  problems  plus  the  others  briefly  outlined  above  pose 
threats  to  both  the  internal  and  external  validity  of  research  investigations 
Careful  design  and  control,  plus  a  good  measure  of  comniQn  senses  are  the  best 
ways  in  empirlcnl  HCudies  to  insure  meaningful  data  from  one-s  research 
e  f f or ts , 
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m    Raanalysts  of  Reaearch  Scudias  in  AVCH 
m  PURPOSE 

A  dQB  Ld-jratum  of  behavioral  scieiice  reHt^arch  is  to  provide  technlcalLy 
corrccC  and  statistically  acciirato  answers  to  all  research  questioas  Inviis- 
tigated  withiii  an  experinient*     Focusing  upon  this  criterion^  researcheirs  In 
the  area  of  Instructional  r,echriology  have  tdencifisd  many  valid  research 
questions,  dasigned  eKperinieats  tlmt  IsalatG  vnrlables  to  be  analyzed,  sslGCted 
statistical  ruodals  Co  study  quantifiable  data,  and  drawn  Inferences  tc  larg- 
er populations  which  were  under  Inves tiga tton .     However,  using  empirical 
observation  mathodology , . ic  was  revealed  that  108  studies  which  appeared  in 
the  first  twenty-five  vQlunies  of  AV  CormimiMHon  Rsvim  selected  a  UTiivar- 
iate  analysis  of  variance  modeL  to  test  eKperirnental  hypothases.     TIwlb^  Che 
primary  purpose  of  study  was  tw-ofoldi     (1)  to  compute  the  inter relattcaships 
between  indeperident  and  criterion  variables  for  the  103  atudles  prevlcusLy 
Identified^     and^   (2)  to  report  chsse  cslculations  in  the  form  perceats  to 
the  profession, 

m  RATIONALE 

A  tandency  exhibited  by  most  researchers  who  published  research  ftnd» 
ings  in  the  journal  cited  hereinbefore    was  to  test  experimental  hypQ  cheses 
based  upon  a  statistical  model  designed  to  Identify  significant  dlfferances* 
Very  few  studies  were     found      th-it  reported  practical  significances  cr  sig- 
nificant relationships,     Clark  and  Snow  (1975,  p,  392)  suggested  that  alter- 
native experimental  designs  should  be  employed  to  Investigate  additloiial 
research  questions,  l,e.,  practical  significances  and  significant  reUtlon-- 
ships,     l^iilG  tnanlpulating  the  design  to  control  variables  is  desirable, 
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tht:  robu'jcnuss  oi  liitiirrGlaeLanshipfl  betwfcan  dependGnt:  and  independent  varl^ 
jbicds  Is  paramotint  to  advanciing  the  kncwladge  hduQ  within  Che  field  ol:  Instruc:-- 
clonal  techno logy. 

e  METHpp 

ReanaLyaing  a  typical  analysis  of  variance  study,   the  rBs&arch^r  at-^ 

compts  to  statistically  suggest  that  there  are  no  differences  between  cYitBCion 

vi^riablss,  while  totally  dLsregardirig  the  statistical  pssociation  b-atv/e^n  the 

iridependent  and  criterion  variables.     Both  statistical  and  practical  sigaifi- 

cancB  tests  can  be  coniputed  from  the  data  raportsd  in  most  F--test  cablea.  toys 

2 

(1953,        325)  suggested  that  a  statistical  Index  called  Cu      (Creak  omfega,  squared) 

can  be  employed  to  estimate  th^  proporidon  of  ustrian^Q  in  J  aaaauntQd  /bn  by  X. 

Oraaga,  squared  values  are  not  difficult  to  Interpret*     They  are  somewhat  film-- 

liar    to  carrelation  coefficients  in  that  each  Is  a  mathematical  mptmBsLcn  of 

the  relationship  that  exists  batween  two  or  mote  variables*     These  values  are 

GXprassed  in  the  forTn  of  a  continuum  ranging  froa  0  to  1*00.     (Omaga,  aqaared 

can  never  assume  a  negative  value.)     mim  cO"  is  1.00,   Che  independent  variable 

2  . 

is  congruent  to  the  criterion  variable,     Hav;avar,  when  LJ    assumes  a  zero  va.^u.^f 
the  knowledge  of  the  Independent  varlabls  does  net  in  any  way  reduce  zhe  uncer- 
tainty about  the  criterion  variable. 

In  most  studies  that  were  reanalyssd,;  nays^   (1963,  p.  327)  forrnula  foe 
computing  CO^  had  to  be  abandoned  because  modt  researcners  reported  iticcni- 
piece  F-test  tables.     Thus,  McNamara's  (1978,        51)  alternate  version  v/as  ap- 
plied to  the  108  experimental  studies  that  appMred  in  the  first  twency-five 
voliines  of  AVCH>     His  equivalent  formula  was  reported  aa 

,.2  ^  (k-l)'F-(k-l)^ 
est.  ^  k-l*F+(N^k)^l 


14^ 


where  F        the  numtiTlcal  value  for  the  ANOVA  te»t,  N  Is  the  total  number  of 


individuals  la  the  sanplfe,  and  k  is  the  nuirnbsr  of  levols  of  the  independen.t 
varriable. 

Thti  following  results  were  gernmne  to  the  study, 

!•     Saven  sCLidles  were  not  included  in  this  teanalysis  because 
the^^  tailed  to  report  incormation  necesaar^/  to  calculata 
CO^.     Ill  all  seven  casess  the  researchers  failed  to  re^ 
pDirt  the  degrees  of  freedorn. 

2,  Two  of  thfi  108  studies  reported  0^  "  values. 

3,  TDfty-aJ.ght  percent  of    the  108  studies  appenrad  in  Voluiiies  19-25 

4,  A  total  of  1^019      tests  ware  cornputed  in  the  108  identified 
studies^ 

5,  About  23  percent  of  all  F-^tests  ireaaalyaed  produced  CO^ 
values  be  ween  ,0001  -  ,0099. 

6,  Si^cty'-eigtit  percent  of  all  CO*^  valuas  fell  within  the  *0001  - 
«0499  Incurenient, 

7t    Only  LiS  peTcent  of  the  F-tests  produced  O}"^  values  larger 
than  .7000. 
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RESEARCH  FOR  THE  PRACTITIONER 

A  eommon  response  when  research  is  mentioned  is  "so 
what"!    We  all  have  read  research  studies  which  address 
seemingly  meaningless  questions  or  which  disappoint  with  their 
inconclusive  or  inaccurate    findings.    This  can  easily  lead 
one  to  be  mistrustful  of  all  research.     To  exercise  caution 
in  reviewing  research  and  in  accepting  the  conclusions  drawn 
is  a  healthy  attitude.     It  is  one  that  insists  that  the 
questions  asked  arft  vai^A  mrsA  +V.-.4.  . 

are  significant.     It  is  one  which  resists  fads  and  seelcs 
truth.     It  is  this  attitude  that  I  hope  you  will  maintain 
during  this  presentation.    However,  to  distrust  and  discount 
all  research  on  the  basis  of  some  bad  experiences  would  te 
akin  to  throwing  the  baby  out  with  the  bathwater. 

What, then,  can  research  do  for  you?    Why  bother  to  come 
to  a  presentation  on  research  for  practitioners?    I  suspect 
that  one  of  the  reasons  you've  oome  to  this  session  -  and  to 
this  Conference  -  is  that  you  are  interested  in  new  ideas 
and  want  to  keep  up  to  date  with  the  field. 

Another  reason  for  an  interest  in  research  may  be  to 
overcome  the  isolation  in  which  many  of  us  work.  Media 
specialists  in  the  public  schools  usually  practice  alone, 
one  to  a  school  and  have  few  contacts  with  others  who  under- 
stand the  work.    Contacts  with  fellow  specialists  are  usually 
limited  to  a  few  in-service  days  a  year  and  regional  meetings 
or  conferences  such  as  this  one.    Knowing  something  about  ' 
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what  is  b^ing  done  ty  researohers  can  help  us  feel  part  of 
th«  larger  whole. 

It  is  also  useful  to  be  able  to  quote  research  findings 
when  proposing  program  changes  to  adminietrators *  Nancy 
Polette,  in  In--S ervice,  advised  using  research  findings  to 
sell  them  on  the  value  of  conducting  in-service 
programs*    They  have  been  trained  to  be  sKepticalf  to  demand 
proof  of  ideas  and  research  findings  can  provide  such  evidence. 

You  can  use  research  to  sell  ideas  in  other  ways  as  well. 
For  the  last  few  years  accountability  has  been  a  by*word  in 
educational  circles  *    With  the  continuing  shrinkage  in  enroll 
ment  (or  stabilisation  in  the  public  schools  at  lower  levels 
than  the  pealc  years  of  the  sixties)!  value  for  dollars 
expended  is  a  significant  concern*    A  study  such  as  Janet 
Simone's  (A  Feasibilitv  Study  for  the  Centralized  Processing 
of  Print  and„  jlon-Frlnt  Materials  for  the  Slementarv  School 
Libraries  In  the  Allegheny  Intermediate  Unit)  could  help 
illustrate  such  accountability  for  an  administrator^ 

The  Allegheny  Intertnediate  Unit  oonsists  of  46  public 
school  districts  in  the  county  of  Allegheny  (in  Pennsylvania). 
The  study  was  limited  to  this  area  and  is  not  generalisable 
to  a  different  geographical  area,  although  the  method  can 
become  a  decision  making  process  for  more  general  application. 

This  study  utilised  the  survey  research  method  with  data 
collected  through  questionnaires  and  telephone  interviews. 
The  data  collected  was^  processed  and  analyEed  by  computer 
using  the  Statistical  Package  for  the  Social  Scienoea*  Cost 
analysis  showed  that  centraliged  processing,  at  the  time  of 
this  study,  would  have  saved  from  $6.52  to  $6.96  per  item 
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over  the  cost  of  individual  district  processing. 

Anothor  area  in  which  research  can  improve  the  return  for 
dollars  and  for  time  invested  is  that  of  selection  of  media. 
When  inappropriate  media  is  selected .  both  money  and  time  are 
lost.    Money  that  should  have  been  spent  on  materials  more 
suited  to  the  task  has  been  misspent.     Time  that  a  student 
spends  using  inappropriate  materials  is  time  wasted.  When 
one  con.^dars  all  that  a  child  must  learn  in  hisAer  school 
years,  there  is  little  room  for  a  Laphazard  selection  of 

.   -  .„™^„^  e,>.iii+^f  of  practitioners  to 

matera.ai.s.     j.u  iinpiu*^  ,j  jr 

select  appropriate  media  for  described  tasks.  Barbara  Dompa 
developed  the  D^Model.     (See  Figure  One.)     This  model  is 
designed  to  facilitate  the  Interaction  of  task,  Btimulus. 
and  learner  components  in  the  selection  of  media  in  order  , 
that  the  messages  necessary  to  permit  the  development  of  the 
learner's  competencies  will  be  discernible. 

As  part  of  this  research  an  experiment  was  designed  to 
test  Whether  or  not  there  would  be  a  significant  difference 
between  novice  librarians  using  the  D-Model  and  novice  librar= 
ians  using  selection  aids  in  the  choices  made.    The  experimental 
design  was  as  follows s 


Pre treatment         Treatment  Post-Treatment 


Phase 


Phase  Phase 


C  S  Cg 

Control  group  ^ 


TP  M 

Experimental  group        .        ^1  ^ 


The  pretreatment  phase  was  a  pretest  administered  to  both 
groups.    The  treatment  phase  for  the  control  group  was  a 
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one-hour  review  of  solGction  aids.     For  the  experimental 
group  it  was  a  ono-hour  explanation  of  the  D-Model,  Directly 
following  the  treatnientp  both  groups  received  a  post»test 
in  which  they  were  asked  to  Identify  appropriate  media  to 
meet  eight  behavioral  objectives. 

Analysis  of  the  data  generated  revealed  a  statistically 
singificant  difference  between  the  perf orTnances  of  the 
control  and  experimental  groups i  with  the  experimental  group 
out-^perf orming  the  control  group.     From  this  it  appears  that 
an  ability  to  use  the  D-Model  can  lead  to  improved  selection 
of  media  for  specific  tasks,  andp  thus,  to  improved  affective-' 
ness. 

Yet  another  use  of  research  is  in  suggesting  methods  for 
evaluating  and  improving  media  services*    In  today's  world 
of  tight  budgets,  it  is  necessary  that  media  specialists  and 
librarians  be  aware  of  the  effectiveness  of  their  programs* 
One  way  to  test  programi  effectiveness  and  to  enhance  aware-- 
ness  of  media  services  is  to  conduct  a  self -evaluation  study. 

In  1973  David  leortsoher  studied  media  center  services  in 
Indiana*    For  this  study  he  prepared  and  administered  a 
quostionnaire  reggLrdlng  services  offered  by  the  school  media 
center.    The  questionnaire  was  administered  to  teachers  and 
media  center  staffs  in  nine  typical  Indiana  schools.  The 
questions  covered  the  areas  of  accessibility,  awareness # 
productionp  instruction  and  consultation #    Media  center  staffs 
were  asked  to  respond,  on  a  scale  of  0  to  3 »  whether  or  not 
a  service  was  provided  and  how  frequently  it  was  provided. 
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Teachers  were  asked  to  respond  as  to  whether  they  received 
the  service  and  to  what  degree  they  were  satisfied  with  the 
service.     In  general,  media  center  staffs  agreed  among  them- 
seives  about  the  services  they  were  offering,  but  the  teachers 
did  not  agree*     This  research  showed  that  In  the  best  of 
circumstancea »  only  about  40  to  of  the  teachers  agreed 

v/ith  the  media  staff  about  the  services  offered,  (Another 
interesting  finding  '    s  that  Individual  differences  among 
teachers  affected  library  use  as  much  as  did  the  subject 
taught* ) 

This  research  indicated  that  a  priority  task  for  media 
Qpecialists  is  to  ensure  that  teachers  and  other  users  know 
what  servicas  are,  in  facti  offered-     One  way  to  alert  them 
is  to  use  a  survey  like  Loertscher' s»     since  answering  a 
survey  can  enhance  teacher  awareness  as  can  a  follow-up 
report.     As  an  outgrowth  of  his  study  Loertscher  has  developed, 
with  Janet  Stroud,  a  service  called  "PSES        Purdue  Self -Eval- 
uation System  for  School  Media  Centers''*    From  a  catalog  the 
specialist  selects  the  survey  Items  to  use  in  the  school.  For 
a  fee,  the  user  will  be  supplied  with  a  reproducible  question^ 
naire  and  machine  readable  answer  sheets  for  each  person 
included  in  the  survey *     The  answer  sheets  are  analyzed  by 
computer  at  Purdue  and  the  results  are  reported  to  the  school* 

In  another  study,  JoAnn  Rogers  studied  the  effect  of 
teachers  knowledge  of  media  skills  upon  their  use  of 
media  centers.     She  developed  the  School  Media  Center 
Fundamentals  Test  in  order  to  determine  whether  "a  relationship 
exists  between  possession  of  necessary  skills,  attitude  toward 
media  formats,  and  frequency  of  use  of  media  center  resources, 
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print  and  non-print."    Her  study  should  have  an  impact  on  the 
proviQion  and  content  of  both  pre-service  training  and  in-service 
training  for  teachers,     The  inedia  specialist  can  increase 
teacher  use  of  media  center  materials  by  providing  approp^ 
riate  in-service  training* 

An  Important  group  of  users  that  should  not  be  neglected 
as  one  strives  to  increase  the  visibility  of  the  media  program 
is  students.     Jacqueline  Mancall  has  studied  the  resources 
used  by  high  school  students  in  preparing  independent  study 


of  student  papers i  by  a  questionnaire  surveying  the  students 
who  prepared  the  papers,  by  a  questionnaire  surveying  the 
teachers  involved,  and  by  interviews  with  the  librarians • 
Mancall  discovered  that  s^tudents  prefer  to  use  monograph 
materials  with  little  consideration  for  the  currency  of  the 
materials,  and  that  materials  other  then  monographs  and 
journals  must  be  specif ioally  brought  to  students'  attention* 
An  oncouraging  aspect  of  the  study  was  that  these  students 
saw  libraries  as  .places  where  they  can  and  do  receive  assistance  * 

Instruction  in  the  use  of  library/media  centers  is  an 
area  that  is  becoming  more  popular  for  research*    Some  time 
ago,  Corle'tt  at  the  University  of  Portland,  measured  library 
skills  as  a  predictor  of  academic  success.     She  administered 
the  Feagley,  Curtlss,  Gaver  and  Greene  library  Orientation  Test 
and  the  Brown-Holtzman  Test  to  a  sample  of  81  college  students 
enrolled  in  freshman  English  classes  in  a  small  metropolitan 
university  (fall,  1969^70  school  year).    The  criterion  of 
academic  success  was  the  grade  point  average  attained  at  the 
end  of  the  semester*     She  perfoCTned  correlation  analysis  and 
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multiple  recressioii  analysis.    Her  conclusions,  from  the  statis 
tical  analysis  of  the  data,  were  that  the  Library  Orientation 
Test  appeared  valid  for  forecasting  success  in  college •  This 
study  should  be  replicated  to  extend  the  population  to  which 
the  findinss  can  be  generalized* 

Also  in  the  area  of  instruction  is  a  study  of  instructional 
rnethodSi  by  Clark  Chle^Yuen  libng^     He  studi.ed  the  effectiveness 
of  two  methods  of  orienting  college  students  to  the  Learning 
Haterials  Center  of  the  Cornmunlty  College  of  Denvert     He  used 
five  groups  with  a  total  of  162  subjects  and  six  librarians  * 
The  criterion  measures  were  a  written  quiz  and  a  performance 
exercise,  both  of  which  had  been  pilot  tested.     He  used  the 
Randomiaed  Kqaivalent  Groups  Posttest  Only  true  experimental 
designs 

ASSIGOTENT  AFTER        ASSIGNMENT  BEFORE 
OR IBNTATi  on  ORIENTA  TI ON 

LECTURE  METHOD 

SLIDE-TAPE 
METHOD 

Groups  1  and  3  received  orientation  by  lecture  i  groups  Z  and 

by  slide-tape  method  *     Groups  1  and  2  received  the  perfor- 
mance  exorcise  assignment  after  the  orientationi  groups  3  and 
^f,  before  the  orientation » 
Major  findings  werei 

1,  Students  receiving  orientation  achieved  significantly 
higher  on  the  quia  than  those  not  receiving  orientation. 

2,  Neither  method  of  orientation  resulted  in  significantly 
different  achieyement  on  the  quiZi 


3*    Students  receiTing  the  assigninen^  before  the  orientation 
adteved  significantly  higher  than  those  receiving  the  assign- 
mont  after  thG  oricntationi 

It  should  he  noted  that  this  study  can  only  be  generalized 
to  community  colLege  students. 

In  1978,  Helen  Qothberg  reported  on  a  study  of  alteinatiye 
methods  of  teaching  "basic  reference  to  ILbrary  school  students* 
The  study  compared  the  lecture  method  to  an  audio-^ tutorial 
method  of  lnstructiDn»    Audio-tutoriaL  units  InGluded  a  recorded 
tape,  a  bibliography  of  required  readings t  suggested  topics 
for  discussion,  adaitional  information  sheets ,  a  study  guide 
and  a  worKsheet.    The  results  of  the  study  showed  no  signif-' 
icant  differences  in  the  understanding  of  theoretical  concems» 
regardless  of  the  method  of  instructloni  jio  differences  in 
the  knowledge  of  the  basic  reference  toolSp  and  no  differences 
in  student  satisfaction  with  the  methodology •    However ,  audlo^ 
tutorial  students  reported  more  satisfaction  with  their  own 
performance  and  kri»wledge  than  did  the  lecture  method  students* 
A  constraint  on  th©  study  was  contimlnation  through  an 
exchange  of  inf ornia.tion  between  the  control  and  experimental 
groups •    Students  in  the  lecture  group  believed  that  the 
audio-tutorial  stulenta  had  superior  noteSi 

In  a  revie'v  of  research,  Williani  H*  illen  analyzed  trends 
in  instructional  media  research  as  reporied  in  AV  Cojmuni cation 
Reviev/  and  the  Review  of  Educational  Research  during  the  five- 
year  periods  of  19^3^73  and  1973^77*    Baaed  on  his  review  of 
the  published  material  coriceitiing  researchi  he  concluded  that 


three  areas  receive  most  attention  from  researchers!  nature 
and  characteristics  of  lacdla  factors  related  to  the  design  a.nd 
developTnent  of  media,  mxid  the  relationship  of  media  and 
media  design  to  learning.    Ke  reported  a  move  froin  evaluatire 
research  to  research  concerning  the  nature  of  media,  particu,- 
larly  that  of  the  pictorial  image. 

Commonly,  aptitude-treatTnent  interaction  studies  have 
attempted  to  isolate  leai-nlng  variables  and  the  effect  of 
anpnoto  of  media  upon  these  variables.     Ann  DeVanoy  Becker 
has  v/ritten  a  provocative  article  concerning  such  metnods  of 
investigation  of  instructional  media  and  its  efrect  on  leariting. 
She  comments  that  the  atimulua-reBponse  model  used  In  suc}i 
studies  introduces  distortion  through  the  experimenter's 
attempt  to  control  the  intervening  variables.    According  to 
her,  the  S-R  model  neglects  the  effects  of  intention,  action 
and  situation  as  variables  in  the  learning  process.  She 
asserts  that  learning  may  be  better  described  through  language 
than  through  numbers  and  the  reduction  of  data  to  statistics . 
In  place  of  the  S-R  model,  BecRer  suggests'  that  field  research 
techniques  bo  used,  such  as  case  Btudies  utilizing  interviews, 
record  or  document  surveys,  direct  observation,  indiract 
observation  and  participant  observation  in  order  to  provide 
more  relevant  data.     The  reliability  of  these  studies  may  "be 
checked  by  ohserving  the  agreement  between  different  ohservers 
at  different  time,  the    agreement  ■between  different  observers 
at  the  same  time  and  the  agreement  between  observations  nadi 
by  the  same  observer  at  different  times, 

I  hope  that  1  have  presented  a  persuasive  case  for  the  value 
of  research  in  helping  us  to  find  answers  to  the  problems 
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that  face  UB  day  by  day.    If  so,  I  want  to  maKe  this  .final 
points    research  is  not  conducted  in  a  vaccuum.  Reseafcher 
need  to  hear  from  the  field.     Practitioners    should  provide 
feedback  concerning  what  research  is  needed  and  concerning 
the  appropriateness  of  the  research  which  has  been  done. 
Only  then  can  the  contribution  of  research  "bft  most 
appreciated  and  utilized. 
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The  ralstionahlp  of  #ye  movements  while  wacching  television,  reading 
abilicyi  Gognitive  ityle,  and  mode  of  presentation  was  studied*  Eight 
five  third  grada  children  ^ere  classified  as  good  or  poor  raadsrs 
CGates^HaeGinltle  Raadlag  Test) .    Cognitive  style  was  measured  by  the 
Chiidran-s  Babedded  Figurai  whleh  classified  aubjeats  as  field  da* 
pendent    or  IndapeBidant.    A  Pol^etrle  Eye  MovCTemt  reeorder  wai  uaed 
to  meaaura  attaticlQn  as  children  viewed  seven  static  and  seven  dynamic 
aegmaota  from  The  glee trig  Compaay>    The  results  Indicated  that  the 
effecta  of  thesa  faccara  varied  fro®  segment  to  le^ent  although  aev^ 
eral  overall  ef feats  achieved  significance.    It  wms  concluded  that 
eye  mov^ent  research  Is  a  uaeful  tool  for  assessing  individual 
differenaea  across  Instructional  modes. 
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Eye  Movement  Research  and  the  Interaction  EeUween  Television  and 
Child-related  CharscteListics 

Lois  J,  Baron 

Research  (Gagne,  1970;  Salomoti  and  SnoWj  1968;  Snow  and 
Salomons  1968;  Salomon ^  1972)  has  stressed  the  importance  of  the 
three-way  Interaction  between  the  medium ^  the  individual  learner, 
and  Che  learning  task.     This  study  eKemplifias  what  Salomon  and 
Clark  (1977)  have  called  a  ■'fusion"'  of  what  had  previoualy  been 
examined  through  linear  investigations  of  main  effects*     The  re- 
search performad  here  can  be  described  in  tams  of  the  definition 
of  media  research  as  the  investigation  of  the  psychological  and 
inatructional  effects  of  media  on  the  responses  of  individuals* 
This  particular  study  describes ,  through  the  analysis  of  eye  move-* 
menCs  measured  during  the  viewing  of  both  dynamic  and  static  tele*- 
vision  presentations  I  the  perceptual  analysis  of  individuals 
possessing  differing  degrees  of  f ield^lndependence-dependence  and 
reading  proficiency* 

In  perceiving  J  the  individual  construcCs  his  world  according 
to  both  the  features  of  the  stimuli  before  himj  and  the  organiz- 
ation of  his  own  coynitivu  Ktructure*     The  gaze,  as  it  is  called 


(Mackworth  and  Bruner,  1970;  Mackworth  and  Morandi ,  1967) ^  is 
.  initially  controlled  by  the  display,  but  eventually  becomes  an 
instrument  of  thought*     Studying  the  eye  movements  of  field  inde-- 
pendents  and  field  dependent  individuals,  and  good  versus  poor 
readers  lends  insight  into  comparisons i  if  anyj  of  developmental 
shifts  from  global  processing  to  highly  differentiated  modes  of 
perception. 

Wolf  (1971)  and  others  (Fleming ^  1969  and  1970)  have  supported 
the  view  that  "the  study  of  eye  movements  allows  for  an  unusual 
opportunity  for  determining  the  reactioti  of  viewers  to  atiinull 
materials."  (Wolf,  p.  113). 

The  research  carried  out  here  was  based  on  the  underlying 
assumption  that  a  child -s  eye  flxationa  reflect  his  cognitive 
approach  to  the  task  and  that  eye  movementi  are  indicative  of 
attending  behavior.     Support  for  such  an  aasumption  li  well 
founded  in  the  literature  (Mackworth  and  Bruner^  1970;  Mackworths 
1967;  Gubaj  Wolf ^  de  Groot,  Knemeyer ,  Van  Attaj  and  Light,  1965; 
Gould,  1967,  1973;  Gould  and  Schaf^jer,  196Sa,  1965b;  Gould  and 
Dillj  1969;  O'Bryan,  1969;  O'Bryan  and  Boersma,  1971;  Noton  and 
Stark,  1971;  Vurplllot,  1968).    The  research  perforrfled  here 
investigated  both  individual  difftrence  and  stiinulus--related 
characteristics  while  at  the  same  time  examlnad  the  Interaction 
between  these  factors*    The  primary  abjective  pf  this  research 
was  to  study  by  means  of  eye  movement  photography  the  attentional 
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processes  related  to  the  organismlc  variables  of  f ield-indepead-- 


to  test  whether  there  was  an  interaction  between  these  factors  and 
the  element  of  movement  in  the  stimulus  presentation. 

Eye  movement  research  into  the  process  of  reading  has  ^^^UA^rally 
found  that  reading  proficiency  varies  with  both  the  ability  of  the 
reader  and  the  complexity  of  reading  material  (Tinker,  1947 | 
Taylor,  1957,  1960;  Conant,  1965;  Fleming,  1969;  Mackworth  and 
Bruner,  1970;  Nodine  and  Lang,  1971;  Edelfat,  1975;  Rayner,  1975) • 
There  is  no  doubt  that  eye  movements  can  provide  a  way  of  ob- 
serving the  relative  effectiveness  with  which  a  person  reads 
(Taylor^  1960),    Without  attending  to  the  stimuli  it  would  be 
very  difficult  to  process  any  information  which  in  turn  benefits 
reading  operations  (Mackworth,  1972).    By  providing  descriptive 
information s  this  eye  movement  research  attempted  to  lend  insight 
into  possibly  different  response  availabilities  between  poor 
readers,  bound  to  simple  decoding ,  and  good  readers,  analyzing  at 
higher  levels  up  the  hierarchy  of  word  and  letter  recognition « 

There  has  not  been  much  eye  movenient  research  as  It  relates 
to  the  area  of  f ield-'independence--dependence  although  two  part-" 
icular  studies  (Conklin,  Mulr,  and  Boersma,  1968;  and  Boersmas 
Muirs  Wilton,  and  Barhamj  1969)  have  found  differences  in  such 
scanning  strategies  as  track  length  and  information  search^ 
Originally  defined  by  Witkin  (1954,  1962,  1964,  1977),  field- 


ence-dependence  and  differing  levels  of  reading  proficieL 
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indepeiidence-dependence  has  been  used  to  describe  Individual 
diffaretices  in  overcpralng  embedding  or  potentially  distracting 
contexts.     Field  independent  persons  possess  a  more  analytic 
approach  to  ta^ks ,  In  contrast  to  the  global  field  approach 
characteristic  of  field  dependent  individuals  (Witkin,  1962), 
The  eye  ffiovemant  indices  chosen  for  this  study  served  as  a  des- 
cription of  those  behaviors  which  are  nomally  attributed  to 
flald  independeni:  individuals  while  at  the  same  time  specified 
the  global  characteristics  of  field  dependent  persons.  An 
Intention  of  this  study  vas  to  assess  the  possibly  embedding 
nature  of  various  televised  stimulus  segments  and  to  exainine 
through  eye  movement  analysis  how  embeddedness  (or  distract— 
ibillty)  may  have  affected  the  attention  of  field  independent 
and  field  dependent  children*    Of  particular  importance  to  this 
study  was  the  ability  of  the  television  code  variable  of  motion 
to  separate  figure  from  ground,     A  number  of  arguments  for  action 
on  the  screen  are  found  in  research  (FowleSi  1973;  Rovet,  1974; 
Allen,  1975) , 

The  appeal  for  research  whose  major  aim  Is  to  examine 
stimulus  characteristics  as  well  as  person^relatad  variables  had 
motivated  this  study. 

Previous  research  comparing  the  various  madia  as  methods  of 
presenting  visualized  instruction  had  been  Inconslstenc  in  its 
conclusions^  although  it  was  generally  acknowledged  that  no 
significant  differences  had  resulted  In  most  studies  comparing 


163 


ERIC 


Che  mif&'^tjJr^         of  two  or  mora  visual  madia  (Dwyer,  1973; 
Jatulsoti,  Suppes  j  &  Wells  ^  1974)*     Salomon  (1974)  suggested  that 
reaear chars  must:  look  at  those  variables  peculiar  to  one  medium 
Itielf  *     The  importance  of  looking  at  the  "s3nnbol  system  unique 
to  each  medium''  (p,  386),  and  how  it  interacts  with  certain 
learner  characteristics  results  in  more  insights  concetning 
individualization  and  ths  nature  of  the  medium,    Loohitig  at  the 
vairious  forms  of  information  representation  and  how  thmy  are 
operaCed  on  by  an  Individual  is  more  baneflcial  than  studying 
differences  across  the  technologies  themselvrii ,    Studies  of  this 
sort  touch  the  heart  of  the  issue  -  "the  relationship  between  the 
way  information  is  externally  represented  and  how  it  is  Internally 
processed'*  (Conway,  1970  ^  p,  159). 

Television  possesses  plctoriaK  ?  .   .^r^.^c,  an.d  verbal  char- 
acteristics (Aliens  1970).    Accordlrii  ^  j-  m  -^  cheorlsts  (Olson, 
1974;  Olson  &  Bruners  1974)5  the  Individual  learner  must  become 
aware  of  the  code  within  a  symbol  system  in  order  to  benefit  froi^ 
instructions*    When  producing  a  symbol  system  code  for  a  particular 
laarner  it  is  important  to  take  into  account  the  possibility  that 
too  many  irrelevant  cues  may  also  hinder  learning*    Rust  (in 
Fovlea*  1973)  reiterated  the  point  that  such  high  appeal  char- 
acteristics  as  rhythm,  rhymes  and  electronic  bridges  tnay  be 
distracting  as  well  as  attention-getting * 

In  looking  at  the  medium  of  televisions  one  would  classify 
movement  as  a  code  variable.    Allen  (1970)  proposed  the  need  for 


studies  comparing  action  and  non^action  within  the  medium.    He  also 
stressed  the  necassity  of  including  individual  aptitude  variables 
in  research  of  this  sort  and  advocated  further  investigation  into 
those  variables  daaltng  with  the  perceptual  processas  and  how  they 
interact  differentially  toward  certain  media  elements. 

The  appeal  for  research  whose  major  aim  was  to  examine 
stloiulus  characteristics  as  well  as  pars  on- related  variables 
motivated  this  study* 

In  the  nature  of  formative  research ^  existing  segments  of  the 
Children's  Television  Workshop  progrmn  The  Electric  Company  were 
eKamined  by  means  of  eye  movement  photography.    By  doing  so 
insights  into  the  nature  of  the  attentlonal  behaviors  of  good  and 
poor  readers,  and  field  independent,  field  dependent  children  were 
observed  in  Interaction  with  televised  segments  from  the  prograni. 
As  wellj  the  nature  of  the  stimulus  element  of  movement  and  its 
resultant  effect  on  attention  was  observed. 

The  independent  factors  of  reading  proficiency,  field- 
indepandence--dapendence ,  and  dynamic  versus  static  Btlniuli  were 
examined  in  terms  of  their  contribution  to  five  eye  movement 
measures      (1)  orientation  time  to  target  or  the  length  of  time 
taken  by  a  subject  before  making  a  fixation  on  a  target  area 
(word  or  sentences)  once  it  appears  on  the  screen  (ORIENT) 
(2)  percentage  of  fixations  on  target  (FIXATION)     (3)  percentage 
of  time  on  target  (TI^ffi)     (4)  lef  t-'to-right  rnovements  or  direct-- 
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ional  attack  (L-#R)  and     (5)  average  length  of  fixation  or 
flKatlon  duration  on  target  (AVERAGE).     These  measures  have  been 
used  In  past  aye  movetnent  research  studies  which  ^  in  being  se- 
lected for  this  research  were  considered  useful  indicators  of  how 
persons  would  interact  with  a  stimulus  presentation  on  the  tele-- 
vision  screenj  (particularly  one  involving  "reading'-)  .  These 
indices  provide  inforraation  as  to  whether  and  how  an  iridivldua^u 
is  directing  his  eye  fixations  to  targets  on  the  icreen,  and  also 
furnish  insight  into  the  affects  of  stimulus  variables  on  eye 
movenients . 

In  light  of  the  literature  canceming  field- independence-- 
dependence,  reading  proficiency ^  mode  of  instructional  present- 
ation j  and  eye  movements ,   the  following  objectives  or  research 
questions  were  formulated: 

1)  By  me aM  of  eye  movsmen^data ;  to  gain  more  insight  into  the 
relationship  between  f ield^lndependence-dependence  and  reading 
prof  Iclency 

2)  By  me^ans  of  eye  movement  research,  to  determine  the  effects 
of  the  media  element  of  mQvement  on  the  visual  patterning  of  good 
and  poor  readei^s  and  field  Independent  and  field  dependent 
individuals, 

3)  By  n^gans  of  eye  movement  research^  to  further  investigate 
differences  In  perception  bstween  good  and  poor  readers. 

4)  U 1 1 1 1  % i_ng_  ey.^nipy^me n t  r e s e a r ch  ^  t o  exam  1  ne  perceptual 

dif f e r en c e s  b e_tw e e^n  f  1  j Id  in d ep e nd e nx^  and  field  dependent  persons^ 
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The  general  objective  of  this  research  study  was  the 
following  I     In  the  _llne  of  forniatlve  o^  Qn^ql_ng  field  research 
using  eye  movements  a^s  a  dependant  vaj^iabley   the^j^search  was 
p^erformed  In  order  to  acq^ulre  more  know ledge^b out  the  g^ommun- 
icatlon jrocess  between  the  medium  and  the  Individual. 

It  was  hypothesiEed  that  good  readers ,   field  Independant 
individuals j  and  dynamic  stimuli  would  ;    _dd  the  more  proficiant 
and  analytical  scanning  strategies  exeii,.  1  if  led  by  significantly 
taster  orientation  times  to  prints  a  significantly  greater  per-- 
centage  of  fiKationri  on  the  target  words  p  a  significantly  '  i?er 
percentage  of  time  on  target,  a  significantly  larger  percentage 
of  directional  attack  on  target p  and  significantly  shorter 
durations  of  fiKation. 

Method 
Sub j  ec  ts 

The  subjects  (Ss)  were  85  third-grade  pupils  (42  boySj  43 
girls)   from  eight  alementary  schools  in  the  Scarboroughj  Ontario, 
Board  of  Education.     The  Ss  were  choseii  according  to  their  com- 
prehension scores  on  the  Gates-MacGinitie  Reading  Test,  Primary 
Form  2  (1964)*     The  test  scores  were  obtained  by  examining  the 
Individual  record  file  for  each  grade  three  student        the  normal 
stream  for  each  school. 

Those  students  scoring  at  least  one  and  ona--haJ  f  standard 
deviations  above  or  bdlow  the  Borough  mean  reading  score  were 
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respectively  assigned  to  good  reader  and  poor  reader  groups ,  The 
Ss  consisted  of  41  good  readetii  (15  boys,  26  girls)  with  a  mean 
reading  score  of  40*82  and  a  mean  grade  equivalent  score  of  5.63, 
and  44  poor  readers  (27  boys,  17  girls)  with  a  mean  reading  score 
of  10s93  and  a  mean  grade  equivalent  scoro  of  1.76. 
Apparat_UjS 

A  Polyraetrics  Eye  Movement  Recorder  (.Model  V'-ll64'-l)  was 
used  to  record  eye  movement  patterns  (EMI^s)   in  this  study, 
Mackworth  (1967)  provided  a  detailect  descript;  jn  of  this  line- 
of-sight  recording  equipment.     UtiliEing  the  corneal  reflection 
method  characteristic  of  this  apparatus,  a  spot  of  light  super^ 
imposed  over  the  stimulus  field  was  produced.    These  eye  spots 
were  recorded  by  a  vide^      mera  as  the  Ss  viewed  the^  *:imulus 
field  on  a  nine-^inch  t^  lf/vL     on  screen  housed  st  e^  .*?el 
approximaLely  two  feet  fuum  the  subject.     The  locatxon  oi  eye 
spots  was  transmitted  as  digital  signals  to  n  computer^ 
Stimulus  Materials 

Segments  from  the  Children's  Television  Workshop  program 
The  Elect ri_c  C ompa ny  (TEC)  were  used  as  a  stimulus  material*  It 
was  necessary  to  produce  a  stationary  presentution  from  the 
already  existing  dynamic   legments  in  order  to  assess  pos.sible  ^ 
differences  in  the  quality  of  eye  movements  between  dynamic  and 
static  modes  of  presentation*     Doing  so  involved  '^freGzlng^'  the 
moving  segments  at  particular  intervals,  juxtaposing  the  fru/en 
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bits  (or  segments)  in  Lhu  same  urdur  as  Lliat:  of  the  (lynamlc  pre-- 
sentation,  and  dubbing  the  ider4tical  voice  track  over  rhn 
resultant  visual^  ,     This  methad  insured  maximum  content  comp-- 
arability  between  the  two  presentations. 

A  dynamic  and  a  static  presentation  wore  created  consisting 
of  suven  "bits"  or  segments  each.     In  the  nature  of  forniative 
research,  the  segments  were  chosen  for  their  individual  stimulus 
qualities  (e,g*  computer  or  electronic  bridges  ^  animation,  dis-- 
tracting  versus  non-distracting  field) . 

ProGiidure 

Administering  the  CEFT 

The^  Children  *s  Embedded  Figures  Test   (CEFT)  was  admin-- 
Jscered  ind    ^idually  to  all  Ss  to  assess  cognitive  style.  A 
standardized  procedure  for  administering  the  test  was  outlined 
in  the  Manual  (Witklnj  Ottman,  Raskin    &  Karp ^  1971),     For  the 
purposes  of  the  analyses  carried  out   Ln  this  studyj  a  mean  split 
was  pevfotTied  to  divide  those  Ss  who  were  considered  field 
independL^ifir  (FI)  from  those  who  were  field  dependent  (FD)  . 
Eye  jnovement  recording  prQccdure . 

Subjects  entered  the  eye  movement  laboratory  individually. 
The  eyti  .^uvement  recording  device  was  explained  to  the  Ss,  They 
were  told  that  the  E  was  interested  in  monitoring  their  television-- 
watching  and  that  by  allowing  the  K  to  do  so^  they  were  contri- 
buting to  the  batterment  of  television  programs  for  all  children. 


The  next  task  was  to  assure  that  eacii  S  was  in  an  appropriate 
and  comfortable  position  for  viewing  the  television  set*  The 
height  of  the  chair  was  raised  such  that   the  S's  forehead  would 
rest  comforcably  on  a  headrest  attached  to  the  equipment,  ti 
bite-bar  was  utilized  to  minimize  any  head  movements  of  the  Ss* 
Each  S  also  had  an  elasticized  £u;eatband  placed  around  his/her 
head  for  the  same  purpose. 

Once  in  a  proper  position  for  watching  television,  calibration 
oi.   the  tfe^cording  equipTnenC  tODk  place.    To  calibrate  ^  Ss  were 
asked  to  look  at  four  cornnr  dots  and  a  conter  dot  affiKed  to  the 
blaiik  t  - « iMv A .U^n  ru;iacn.    The  recording  equipment  was  then 
approp  >  i-^     i  y  aa  j?*sted  , 

Each  subject  viewed  the  two  bLxmulub  preaentations^-dynani i 
and  stati".     The  order  of  presentation  for  each  S  was  previously 
determined  with  the  aid  of  a  table  of  random  numbers,  Retv/een 
presentations  each  S  waF  given  a  two-minute  rest* 
S  c  Qt  1  nj^  J  r  o  c  e  du  r  e . 

As  outlined  previously,  five  eye  movement  measures  were 
chosen  for  investigating  the  quGstions  and  hypotheses  of  this  t 
research  study. 

Before  measuring  the  eye  movements ^  It  was  necessary  to 
specify  the  criterion  for  defining  an  eye  spot  as  a  ''fixation'' 
for  the  purposes  of  this  stiidy*    Taylor  (1960)  points  out  that 
the  averags  length  of  fixation  fc:  a  third-giader  is   ,28  seconds 


(or  280  miilititiconds)  per  eye  spot.     Ot./ing  to  the  fact  that  sub^ 
jects  in  this  expariinant  included  children  reading  at  levelf>  as 
low  as  grada  one ,  it  was  decided  to  establish  .10  seconds  or  100 
milliseconds  as  the  minimum  amount  of  time  necessary  for  an  eye 
spot  tw  :  '        .  ideL'ed  a  fixation, 

Befo4c  analy.^^ing  the  data,   it  was  necessary  to  iasure 
validity  cf  the  me nsnrements  by  eliminating  defective  recordings. 
Upon  compietiug  the  data  gathering,   the  video  tapes  of  each 
subject's  EWb  wern  reviewed  without  knowledge  of  the  character- 
istics of  that  particular  subject.     Those  segments  lacking  in 
callbrationj  as  indicated  by  the  eye  spot  being  off  thfj  center 
spot  of  the  screen  following  the  termination  of  a  sep.ment,  were 
excluded  froni  anal  '^sis.     This  elimination  proces;i  Ipd  to  a 
possible  situation  in  which  for  one  particular  subject  there  may 
have  been  only  one  or  tvo  good  segments  from  which  data  could  be 
analyzed,     The  program  used  in  the  final  analyses  comprised  the 
necessary  measures  for  the  handling  of  missing  data* 
Preparation  of  D a t a  f  o y  Jlna ly sx s 

In  o  I     asRG8S  the  eye  movement  patterns ^  information  as 

to  the  exa  ae  and  location  of  each  target  was  provided  as  data 

to  the  PDP-9  computer.    The  targets  within  each  stimulus  segment 
were  a  word  or  group  of  words,  and  the  target  field  in  which  the 
eye  mo.'vment  measnrcmentb     jrc  made  con^i^^ted  of  a  four-^sided 
figure  su^rotinding  these  words ,     TargetL   i;;  the  dynamic  segments 


consiited  of  either  targetting  certain  areas  during  the  present- 
acion  whertt  a  target  letter  or  word  would  lio  at  a  particular 
point  in  time  (e.g.  Try,  Dry,  Shy)  or  targetcing  a  larger  area  in 
which  whole  words  or  sentences  would  appear  (e*g,   the  Princess  and 
the  Frog)  ,     Targetting  the  static  segrnents  consisted  of  a  similar 
process  in  which  the  urBnoving  words  or  letters  were  targctted  for 
their  position  on  the  screen,    Targetting  constituted  transmitting 
a  pulse  from  each  corner  of  a  target  to  be  changed  to  numerical 
coordinate  inf ortiiation  in  the  computer  (in  the  same  manner  as  an 
S's  eye  spot).     Once  this  information  became  computer  data,  the 
location  and  analysis  of  subject  data ^  already  stored  on  computer 
tape,  was  carried  out  on  the  eye  spots  occurring  within  the  time 
of  target  appearances.    With  the  time  and  coordinate  information 
avallaL.le  within  "he  cnmputer,  a  seri'ss  of  programs  reduced  the 
data  in  terms  of  the  five  eye  movement  Indices  of  Int^^^^-t  to  this 
study—orientation  times   (in  thirtieths  of  a  succnd)  ^  .  iKations  on 
target  (as  a  percentaBe)  ,  time  on  target  (as  a  percentage)  , 
average  duration  of  fixation  (in  thirtiethp  of  a  second) ^  and 
lef  t--to--right  (L— ^  R)  movement  (as  a  percentage).     The  eye  Liove- 
ment  inf omiation  was  identified  and  categori?.ed  for  each  segment 
and  subject.     In  the  final  analysis  only  data  from  the  stimulus 
first  presented  to  a  subject  (dynamic  or  static)  was  utilized  in 
the  calcuLanions. 

Statistical  Treatmcn^  of  Data 

#  -■  --  =  ^  '^^-^^         —  —  —  — - 

A  regression  prQcedure  was  used  specifying  the  analysis  of 
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variance  model*     The  particular  prugram  utilized  wac^  taken  from 
tha  Statistical  Analysis  System  (Servicu,  197^), 

The  program  applied  Che  mcithod  of  lecist  squares  in  fitting  a 
linear  model  to  the  data.     The  variation  attributabla  to  each  of 
the  independent  variables  in  Che  model  (treated  as  classification 
^'ariables)  was  axaminud  as  were  the  interaccions  of  these  variables 
on  the  five  denendent  or  eye  movement  m-     ;  jres.     Traating  the 
variables  as  Independent  of  each  other  was  of  concern  to  this  study. 
For  this  reason ^  the  partial  hu:;.  of  squares  was  Che  source  of 
variation  examined.     The  data  analysed  was  that  available  from  each 
subject  classified  under  either  the  dynamic  or  acntic  condition^ 
good  readet  or  poor  reader  category ^  and  fle^d  Independent  or  field 
dependent  classification. , 


Regression  analyses  were  calculaced  for  eacl;  segment  (referred 
to  as  ALL)  as  we]  1  as  an  overall  analysis  avernged  over  all  segments 
togetiier  (MODE=AVLRAGE) .     To  accounc  for  the  possibility  th^t 
dif f erenci^r^  or  lack  of    iifferences  in  the  dependent  measures  could 
be  due  to  long  exposure  to  a  stimulus,  a  regression  analysis  on 
informaiion  gachered  during  the  early  seconds  of  a  sugmont's 
eKposure  was  executed  (INIT), 
1 )     Otientation  Time 

Mode^Average  data  gathared  throughuut  the  segments  r^^vealed 
the  .only  significant  dlfferencu.-  for  this  measure*  Good  readers 
did  orient  significantly  faster. than  did  poor  readers   (F^  5,07; 
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p^dS).     significant  diffei/ences  weru  found  bedwis.m  fi'j  g 
indepandent  and  field  dc.pendent  subjfcci:s  (F— ^*M;  p  <  ,Uj)  and 
between  inalas  and  females  (F=5.49*  p^.05).     Field  independant 
subjGcts  oriented  to  the  target  woiJc  significantly  faster. 
Miilo::    ;lso  possessed  significantly  faster  overall  orientation 
tlr";jb  . i  1    iiough  in  one  particular  cartoon  segment  girls  orionted 
significantly  faatur  to  the  target  wordn  (Fs4.9^; 

Orientation  time  measures  yielded  three^-way  interaction  and 
only  on  averaging  data  over  all  segments,     Tlie  interaction  of 
CEFT  X  SEX  varied  signif icantly  with  the  presentation  mode 
(F^4,63;  p<^.05)*     The  dynamic  mode  yield  the  shortest  orien- 
L  'f  ton  time  for  both  field  dependent  boys  am-  girls* 

The  inte^ction  of  READ  x  CEFT  x  SEX  was  also  significant 
(F-:^  6,30:  p^.Ol;*     Both  malG  and  female,  field  dependent,  good 
rt:adcro  had  the  shortest  mean  orientation  time* 

Modt2  of  presentation  also  varied  significantly  with  the 
combination  of  both  READ  x  CEFT  (F=3.69;  p<.05)   and  READ  x 
SEX  (F^3.17;  p^-05).     Shorter  orientation  times  were  disco- 
vered for  field  dependents   p,ond  readers   in  the  dynamic  condition ^ 
while  female  gcow  Leaders  aljo  eKhibited  the  shortest  times  in 
the  dynamic  condition, 
Flxa  t ion 

Significant  differences  in  percentage  of  fixation  on  target 
between  field   indupendent  and  field  dupenduiiL  subjucta  were 
discovcrud  in  one  segment  (Fit  4*07;  p<,05)  winjre  field  indepen'^ 


clients  wurc  characturlzau        linviiig  moru  £i>::iLLuua  on  tairgut. 
Data  for  mode  of  prestintat ion  ulso  conflrmud  tliu  hypothusis 
on  another  segment  for  both  ALL  (F^4.57;   -f.OS)  and  INIT 
(F^6,34;  data.    The  percentage  of  fiKacions  on  target 

ware  not  subBtantially  differanct  across  modes  for  the  other 
segmQuts  and  averaged  over  the  segrnerts.     Significant  differences 
did  occur  between  mai^  and  female  on  one  segment  (different 
than  the  above  two  segments)  for  ALL  p  ^,05)  and  INIT 

data  CF*^4.80;  p^.05).     In  both  cases  girls  fixated  on  target 

wurcg  eignif; cantiy  more  than  did  boys. 

One  segment  uncovered  a  READ  x  CEFT  x  PRES  interaction 
(AI.L  -  F^5.66;  p^.03)  with  the  combination  of  poor  reader  k 

field  independent  x  dynaniiCj  demonstrating  the  largest  percentage 

of  fixations  on  target.     Furthermoraj   two--way  READ  k  PRES 
':.H3\  p<»05)   led  to  significant  findings  when  data  was 

..r.-t:..\^ged  for  the  segments  taken    together  and  only  on  INIT  data. 

Time 

Data  for  two  different  segments  (one  for  ALL  datd  F^5.71; 
,05  the  other  for  INXT  data  F^5.57;  p<.05)  resulted  in 
si|inif icantly  different  pcrec^ntages  of  time  on  target  between 
good  and  poor  readers.     Both  rusults  sui>purtud  the  hypotriuHis  * 
On  e):ai-iining  Mode-Average  data  the  hypothesis  calling  fur  reading 
proficiency  differences  on  this  measure  was  not  coi^iirmed. 

Significant  differences  between  ;:lie  dynamic  and  static 
segments  were  revealed  for  ALL  iata  on  ontr  segmunr  (F^  8*65; 
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p^.Ol)  .     In  this  parclcular  case  it  way  ciiu  :^Lacic  rachur  than 
dynamic  presentation  that  reBultud  in  moru  timu  spent  on  the 
target  words. 

Mode^Average  dataj  on  examining  whole  segments^  resulted  In 
significant  male/feniale  times  (F^4.95;  p<.05).     Girls  spent:  a 
significantly  greater  percentage  of  timu  on  targets  than  did  '-i  '.'s* 

CEE'T  K  PRES  was  significant  for  one^segment  (F^  10*08:      :  .  :\)  \ 
with  the  combinations  of  f ield^independent  static  and  fiela- 
dependent  dynamic  resulting  in  greater  percentages  of  time  on 
target , 

Left-To-Right  Directionality  (L-^R) 

Significant  results  in  an  opposite  direction  than  that 
predici  o.d  were  found  in  early  data  for  one  particular  segment 
CF^11.4^;  p<  *Q1)  *     As  well^  early  data  for  the  same  segment 
revealed  that  there  was  a  significantly  larger  percentage  of  left^ 
to— right  movement  in  tho  dymanic  presentation  (F^  8,65;  p^.Ol)* 
Average  Duration  of  I^ij<£^tion 

In  both  ALL  data  for  one  particular  segment  (F^  4.46;  ,05) 
and    averaged  across  segments   (F^5.18;  p<,05)   field  independent 
subjects  possesrod  larger  flKation  durations.     Early  data  for  two 
different     segnients  showed  that  the  st.itic  as  opposed  to   the  dynamic 
presentation  did  indeed  lead  to  larger  .    lation  durations  (F=.  6*35; 
p  < .05)  in  one  segment,  while  boys'  fixaticns  on  target  were 
si^;^!! icantly  larger  than  girls'  in  another  segment  (F^6.60; 
p<.05).    ALL  data  on  one  particular  cartoon  aegment  lead  to 
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significant  CEFT  x  PRES  (F^  5.34;  p<.05)  and  READ  x  PRES 
(F3  3.83;  p<.05)  interactions.     CEFT  x  PRES  was  also  significant 
fur  Modc'-Average  dcM..  (ALL  -        3.80;  p<.05).     In  both  cases 
the  shorter  fiKat^^.^     tirations  were  characteristic  of  field 
independent  sub j     -s   .le.Ing  the  static  condition. 

IscjLisgjon 

The  folluw^n:^   Is  n;    Interpretatldn  of  the  results  as  cricy 
relatL  to  the  more  general  research  objeccives  askod  of  this 
study. 

1  •     3y  means  of  aye  movemQnt  data,   to  gain  more  inf  ^j?ht  Into  the 
relationship  between  field  articulation  and  reading  ai^llity . 

Past  research  has  indicated  that  f leld-independence--depen= 
dence  was  not  closely  related  to  the  verbal  component  of  reading* 
From  the  perceptual  analysis  perfomad  hercs  there  was  very  little 
evidc  Acu  to  sugge'^n  that  f  ield-^indepandencQ-dcpendtince  was  related 
to  Lb^  ^verceptuai  =  ompon^nt  of  reading  proficiency  (at  least  on 
the  tyr  movement  measures  used  to  describe  reading  proficiency  in 
this  research)*     Although  there  may  ba  comiriunallty  on  other  mea- 
sures between  those  leaning  toward  high  reading  proficienr.y  and 
field  Independency  5  eye  movement  data  In  this  study  revealed  lie  tie 
evidence  to  demonstrate  that  good  readerH  wuru  good  readGrs  because 
they  possessed  a  field  i  id^pendent  cogniuive  style*  Naturally 
such  an  influence  is  limited  to  the  eye  movement  data  in  this 
particular  study,  but  it  does  raise  thu  quu^tion  as  to  what  in  the 
link  between  the  two  variables. 
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i •     By  maans  of  eye  inovement:  rcHgarchy  to  duturmlne  tha  ijfftic_t^B 
of  the  media  elefi.Gnt_of  moyuTne nt  on  the  visual  pattern tng^£_gQO_cl 
and  poo readers  and  fj.t-j^d  Independent  and  field  dependent 
individuals . 

The  med  la  ulGmen  t  s  tudied  lie  re  waa  thu  dynamic  versus  s  tat  ic 
quality  of  the  medium  of  television.     It  had  been  purported  in  the 
literature  witli  little  supportive  research  that  tho  dynamic 
qualities  of  television  were  a  useful  aid  in  attracting  and  main^ 
tainlng  attention  to  the  screen.     Gross  (1974)  and  Allen  (1975) 
believed  In  thcd  r^LiSi^ntial  information  -  reidying  nature  of  the 
code  of  instruction.     Studying  the  dynamic  versus  static  quality 
was  essentially  an  investigation  into  the  code  of  the  medium  of 
television. 

It  appears  from  the  uata  that  in  a  rather  complex  stimulus 
presentation  the  dynauil'^  nic^le  did  prov.'  to  be  beneficial  in  the 
cose  of  a  few  of  the  eye      ■ .r^'^.L  'n^.  "Hii:;es  while  in  another  more 
slow-paced  animated  segment  the  static  presentatiDn  l^ad  to  more 
time  being  spent  on  the  target  words.     In  another  segment  in  which 
sentences  wero  formed  in  a  lef     ^  ^^-right  manner  by  means  of  ani- 
mation, left^tu-- right  scnnning  patterns  wire  reinforced. 

Generally,   the  visual  cueing  characteriHt Lc  of  dynamism  within 
tii^?  medium  of  tulevisjon  did  not  result   in  many  significant 
differences  in  viewing  patterns  as  measured  by  the  eye  movement 
Indices  of  this  research.     The  auditory  component  of  the  stimuli 
may  have  diminished  dif  f ^^-ennos ,  for  as  Mock  (1975)  indicated ,  the 
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auditory  channel  is  a  vary  HLrong  cua  an  to  v;huru  Co  look* 

The  general  lack  of  yigiilCiCcint:  findings  in  IntaractJon  with 
preaencatlon  mode  asems  to  negate  the  possibility  of  visual 
supplantation.     The  results  of  tnis  research  suggests  that  move-- 
ment  was  not  ntzcessarily  an  essential  aid  to  particular  rtiading 
or  f ield-independant-dependBn t  groups.     Information  as  to  whore 
aigniflcant  diffareiices  and  interactions  did  in  fact  occur  as  a 
result  of  movement  or  lack  of  it  should  be  eKploited  as  us 
of  instructional  assistance  to  particular  viewers. 
3 .     By  m e a n s_  □  t_Qy_<B  rno ve man^  r e s r c h_ ,  to  f  u r t h u r_  1  n v es^tl g j te 
d i j Ce renccB  in  pergeption  b e_t we e n  go o d  an d  poor  _r e ad e r s  . 

The  results  suggest  that  readcrB  were  only  dif f ercnciated 
in  the  presence  of  particular  qualities  of  a  stltnulus  segment. 
As  Rayner  (1975)  and  Conant  (1965)  believed,  it  would  appear  that 
text  not  individuaj   hypothesis  testing  detormlned  eye  movement 
patterns* 

Questions  which  come  to  mind  ay  a  result  uf  the  analysiH 
include  the  following^     V/as  supplantation  ref?ponBible  for  matching 
tlie  eye  mo/ements  of  poor  renders  with  those  of  good  readers? 
Was  It  good  readers'   ''sampling"  (Wiener  and  Cromer,  1967)  o[  the 
tarn^et  that  resulted  In  similar  eye  movomoncs  to  those  of  no  or 
readers  in  many  instances?    Were  the  HtimuIL  nor  c:LCJting  onougb 
to  hold  the  attenCion  of  good  readers?    Were  some  of  r  -i  segments 
not  visually  complex  enough  to  allow  for  differences  between  the 
two  groups  ?     Research  qucsCionB  such  as  tiiese  could  be  Gxamined 


by  stiixtly  producing  segments  guarad  toward  such  an  investigation. 

The  reruLts  of  this  eye  movement  study  did  not  support  pre- 
vious claitna  that  good  readers  exhibit  shorter  durations  of  fixa^ 
Cion  vu.^n  reading*  In  more  cases  than  not,  they  not  only  paused 
as  long  when  on  target  but  also  spent  more  time  on  the  words  than 
did  poor  readers.  The  findings  related  to  Che  effects  of  dynamic 
versus  sLaLic  stimuli  also  seem  to  weaken  the  supplantat ion  argu-^ 
ment  as  it  relates  co  readers. 

The  results  of  this  study  point  toward  stimulus-specific 
qualit     s  of  particular  segments  that  resulted  in  any  differences 
between  trie  two  reader  groups.     Further  analysis  :nto  Isolated 
e lenient s  of  thtse  segments  would  yield  information  as  to  the 
possible  occurrence  of  common  eltjments  that  dif f erGntiated  the 
groups.     Knowledge  such  as  thib  would  be  useful  to  instructional 
media  designers, 

4 ,     Utilizing  eye  movemen^^e^Sjarc^l^^o  eKamine  the  perceptual 
difjarences  between  field  Indepei '  ' ent  and_£j-!d  clep en d e n t  persons . 

Siniilar  to  the  results  related  to  reading  rroflciencyj  tliere 
existed  rather  segmont--specif ic  eye  movement  differences  invol- 
ving field-- independence^dependence .     In  one  particular  segment  and 
Cor  Mode-Average  data,  field  dependents  did  pause  signiC icantly 
longer  when  on  target  as  they  made  an  at^iumpt  to  decode  the  words. 
These  results  were  supported  in  the  literature  by  claims  of  cogni- 
tive delay  and  the  global  field  de pun a ent  style  of  processing, 
while  the  shorter  durations  were  more  characteristic  of  the  focusing 


in  amlyciq  mo  da  of  field  IncliiiionciG^it   Indiviclu  ^iLs . 

Thii  quest  Aoi%  nrises  hore        to  whcttnar  tli  c  lack  of  aignificaiit 
di  ff  evancea   Is   ca  us^d  by   individual  or  scimulufl  properties.  Ara 
qualities  0^  ttie   stimulus  aupplantiiig  cperatlons  for  field 
de^^Grtdeiits  cPr  tor  Ing  fieLd  iiidepend^ntsf    i\re   the  stimuli  not 
emfcadditig  eciough   to  dlffarontlatea  b^tv^em  the  groups? 

The  sh^rt^r  cluiratioas  oi  flHatioa  iniVQlviEig  field  independent 
subjeets  Cacl^as^  ffor  oae  segment  ^nd   tor  Mode-A\^erage  data)  seem 
to  gLiPport  ^  field  indepandetit  a^mpiitig  process  chat  suggests 
tliat^  chara^te^is cic       tlielx  mtyu  of  processdiig  information ^  field 
IndcpendGnt  incJividualP  Bpetid  tltne  ^ornpa^lag  target  with  non  target 
areas  as  th^y  attempt  to  find  mGanifig  in   fl  stiaiulus  compleK* 

It  appsar#  fffoiti  ttxla   Btudy  that  the   task  and  its  Inherent 
<luailltie§  w&ra  vhMt  Jiade  light  of  the  cog  ni  tive  style  differences 
Indivldunis  po^ae^sad.     FufthGr  s  tu^y        tliaee   atlniulus-speclf  Ic 
^lemelits  %b  mm  dtrfection  wheta  fiitt^rD   re  ss^irch  could  take  aim. 

Oi   tl^e  <|Li^sttoas  pceed  by   tli  is   re^eaTcbi  the  one  which  stands 
out       th^t  %^ht^h   asks  whether  ey^  r^ov^-mesitg  are  stiinulus  or 
persofi-sp^cliic  •     Th^  r^ssareh  pa^f^rnied  liere  ild  lend  insight  into 

pyoblew  as  tnatn  affeccs  and  l^t& Tactions  w&re  examined*  The 
data  Indlc^ites  tlio^t  indiVLdua.!  qualttiesQf  eaa  li  segment  played  a 
large  part       vrtief  e  the  sulUeccs  ^er«  Looking*     Further  study  of 
chese  St  iniultis  specific  eLements  is  ^ne  direction  where  future 
ires^atfch  could  Calcc  .^lm*    is   Olson  (  1974)   and  d  thers  suggested. 
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Wd  muBt  reauarch  in  tlupth  thu  aodu  or  yyiitax  u  I!  partJuuiair  muaiuiii, 
More  knowledge  should  be  ascertainQd  as  to  the  effects  of  .specific 
cueing  devices  J  particulatly  audicory  cues.     Research  comparitig 
liMs  j  eliminating  the  voice-over,  would  no  doubt  yield  worthwhile 
Inf ornLatlon  as  to  the  eP£<^iCU3  of  the  vinual  cues*    The  auditory 
component  may  have  beeTi  the  sole  channel  from  where  some  of  th^ 
subjects  rGceived  info rMt ion  that  assisted  thein  in  analy?.ltig 
the  field. 

From  the  results  cjf  this  study  it  appeafed  that  in  more  cases 
than  not  the  dynamic  stimulus  did  not  make  a  difference  in  the  eye 
Tnovementa.     If  an  asymptotic  level  was  reached  ^here  the  dyriamic 
atimulus  did  not  serve  as  a  cueing  aid^  what  was  that  level?  Further 
research  ought  to  concern  Itself  wich  such  a  qtiestion*     In  other 
vords>  was  there  a  level  at  which  dynamism  ceased  to  be  beneficial 
und  became  an  inforniatlon  processing  hlndranca? 

Further  study  of  E^ffs  as  they  relate  to  cognitive  style  variahles 
ought  to  be  performed.     Resultant  informatioti  wuld  be  usaful  to 
instructional  designers  and  those  researchers  exploring  the  area  of 
cognitive  style.    Other  organiBrnic  variables  includini  anKiGty, 
atage  of  development  (a  la  Plaget  or  Bruner)  ,  or  locus  of  concrol 
could  be  included  In  Such  Investigations. 

Although  many  studies  escist  in  this  area ,  complete  infoTm^itlQn 
concerning  eye  movementi  and  reading  has  not  teen  totally  uncovered. 
Recent  research  (Mitrani,  in  Edelfat,  1975)  has  suggested  that  good 
readers  ''read"  during  saccadic  EMs*     Information  such  as  this  is 


useful  in  interprijUng  rusults  althoujjh  lurthur  oKploracion  is 
needed  in  this  area  to  substantiate  his  fLndings. 


between  ALL  and  INIT  data.    What  happenod  to  the  attention  of 
subjects  as  they  viewed  a  segmGnt  over  time  was  a  factor  that  must 
be  taken  into  consideration  when  studying  individual  differences 
and  different  modes  of  instructional  pr£!Sentacions ,  Further 
research  into  the  most  beneficial  length  of  time  for  the  eMpoyure 
of  a  stimulus  presentation  ought  to  be  invcscigated  keeping 
attentional  factors  and  pacing  of  a  segnient  in  mind* 

Finally s  sen  differences  as  they  relate  to  eye  movements,  the 
medium  and  organismic  variables  ought  to  be  Investigated  further. 
Thla  study  produced  some  interesting  main  and  interactive  sex- 
related  effects*     However,  it  vas  one  of  the  few  Ehi  studies  to  have 
done  so.    Many  organismic  vari^ibles  have  bean  correlated  with  bbk, 
but  it  is  rare  to  find  an  EH  study  which  not  only  dlf f erenciatcd  the 
"looking"  behaviors  of  males  and  females ^  but  also  correlated  these 
EMPs  with  other  measures. 
Educg  t  ignal  Inipl i^a t ign^ 

Implications  resulting  from  EM  data  surrounding  organismic 
variables  as  they  stand  alone  and  in  inceractlon  with  other  factors 
related  to  the  medium  of  television  are  useful  to  those  involved  in 
instructional  design*  Application  of  theory  into  practice  need  not 
be  made  solely  Co  the  medium  of  televislDn^  but  qan  also  be  applied 
to  other  methods  of  communication  (inciuding  the  teachur)  that  are 


Not  specifically  comparGd,  varying  results  wejre  acquired 


avaiiable  in  thu  Ccachlng-lciarning  uiiv  i  ronnitMit  .     Knuwing  that 
quaiicies  of  a  stimulus  fiiiid  mny  ovoka  certain  EM  responsea  in  in^ 
divlduals^  wich  a  furuher  invuscigatioa  into  the  elemonty  of  these 
stimul-ii  Instructional  packages  can  not  only  be  designed  so  as  to 
take  advantage  o£  these  attention^gat ting  and  rnaintaining  devices, 
but  can  also  be  paced  according  to  the  perccptucil  style  of  infor-- 
matlon  processDrs,     Although  it  was  found  that  the  dynamic  modQ 
prodiiced  positive  effects  for  some  individuals  while  negative  results 
for  others,  geaeraXly  there  were  no  significant  differences  due  to 
mode*     Information  such  as  this  is  usaful  to  those  who  have  only 
looked  at  movemerit  on  the  screen  as  a  positive  characteristic  of  the 
medluin.     Instructional  materials  designed  for  certain  individuals 
might  best  be  produced  avoiding  what  my  prove  to  be  hindering 
devices  * 

ila  the  National  Institute  for  Education  Report  (Gibbon  et  al  ^ 
1974)  suggested  J  EM  studies  are  a  useful  "real-- time"  indicator  of  the 
perceptual  proceBses  of  certain  individuals.     Eye  movement  studies 
allow  one  to  study  the  style  of  individual  prqcesaors  and  to  inves-- 
tigates  as  in  this  study ,  the  effects  of  these  styles  under  the 
influence  of  certain  environmental  factors.     There  are  few  dependent 
variables  that  can  produce  such  useful  Infarmatiori  as  it  relates  to 
indivtduql  differences*     Individual  differences  are  a  major  concern 
to  educators  as  they  investigate   '"person''  qualities  interacting  t^ith 
various  environmeTics  and  instructional  methods*     Knowledge  of  Infor- 
mation processing  skill  gained  from  F^M  studies  is  useful  Co  teachers 
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and  producers  of  instructionLii  tolyvLMion  wlio  sot:  up  tuciching-'laarnlng 
environments  while  keeping  in  mind  an  ins trLict  lonal  pace  suitable  to 
various  learners. 
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PRINCIPA.LS*  PERCEPTIONS  OF  ACTUAL  AND  IDEAL  ROLES 
OF  THE  SCHOOL  MEDIA  '  UIST 


Introduction 

Schaol  media  centeJ^s  have  been  undergoing  changes  in  the  last  few 
years.    Dne  result  of  these  changes  is  that  school  fnedia  centers  have 
becofne  the  heart  of  the  instructional  program.    The  new  functions  per- 
formed by  the  school  media  specialist  focus  heavily  on  serving  the  needs 
of  the  professiofial  stiff  and  the  needs  of  the  learner.    Because  school 
media  center  activities  are  closely  integrated  with  the  educational 
program  teachers ^  administrators    and  principals  need  to  understand  and 
support  the  center  and  the  functions  of  the  school  media  specialist.  The 
most  Important  non-niedia  professional  to  the  functioning  of  the  media 
center  is  the  principaL    It  is  the  principal  *s  enthusiastic  support^  or 
lack  of  it^  which  often  determines  the  success  or  failure  of  programs. 
If  the  principal  supports  the  media  specialists  and  their  program  the 
center  has  great  potential.    Without  this  interest  and  understanding,  a 
significant  media  prograiTi  1s  Impossible. 

Purpose  of  Study  and  Sampje 

Because  the  principal  is  a  key  force  in  the  media  center  program^ 
this  stud>  chose  to  describe  the  perceptions  of  the  principal  toward  the 
role  of  the  school  media  specialist.    Two  hundred  high  school  principals 
in  the  state  of  Iowa  were  chosen  to  make  up  the  sample.    The  randomized 
list  of  tw  hundred  principals  was  obtained  from  the  Iowa  Department  of 
Publ lc  Instruction . 
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This  study  focused  on  thr^e  general  questions: 

1)  What  perceptioris  do  principals  have  toward  -the  ideal  r^les  and 
functions  of  the  scrool  rnedia  specialist? 

2)  What  perceptions  do  principals  have  toward  ^he  actual  jerformance 
of  the  school  ruedla  special  1st  1 

3)  Is  there  a  significant  di  ffererce  betv/een  tre  actual  a^d  ideal  role 
of  the  school  media  specialist  as  perGelved  by  seconda ry  school 
principal  s? 

In  addition  to  studying  the  perceptions  of  prii^cipals  as  a  whole^ 
the  principals  were  grouped  according  to  the  fonow^ng  variables  and 
di  fferences  were  studied^  ' 

1)  Those  v^ho  were  ennplc>yed  In  a  small  ^enrollm^nt  of  5O0  cr  less)  or 
large  ( enrollment  of  501  or  more)  schools 

2)  Those  vvT ho  were  einpl^yecl  fn  rural  (populatiori  otSiOPO  er  less)  or 
urban  (population  of  5  aOOl  or  more)  geographical  area, 

3)  Those  v/ho  had  a  fned-f  a  course(s)  or  tho5e  whc?  h9d  not  had  a  jiiedia 
course(  s} . 

4)  Those  who  had  been  ^ni|iln\      in  the  schwl  (-Cehure)  for  0-5  years^ 
6-10  years,  n--15  y^ars,  or  IS'-niore  ye^rs, 

The  data  col  lected  was  used  to  test  the  fonowing  hypotheses: 

^1)  There  Is  no  signi  fi^ant  difference  between  the  actual  and  ideal 
perfcrmance  of  th^  school  media  specialist  m  perceived  by  high 
school  pri  ndpals  , 
*2a)  Ther€  is  no  signi  ficant  difference  bstwen  prlncf  pals  -tfron)  small 
and  large  schools  in  thei  r  perceptions  of  what  sNoul  d  be  the  actual 
role  of  the  scliool  niedia  specialists 


2b)  There  is  no  significant  dHference  between  principals  from  small 
and  large  schools  in  their  perceptions  of  what  should  be  the  ideal 
role  0"f  the  school  media  specialist. 

*2c)  Therd  is  no  si gni fi cant  difference  between  actual  and  ideal  per- 
formance  of  the  school  media  specialist  as  perceived  by  principals 
froin  small  schools. 

^2d)  There  1s  no  significant  difference  between  actual  and  ideal  per- 
formance of  the  school  media  specialist  as  perceived  by  principals 
from  large  schools. 
^3a)  there  Is  no  significant  difference  between  principals  from  rural 
and  urban  schools  in  their  perceptions  of  what  should  be  the 
actual  role  of  the  school  media  specialist, 

3b)  ftera  is  no  significant  difference  between  principals  from  rural 
and  urban  schools  in  their  perceptions  of  what  should  be  the  ideal 
role  of  the  school  media  specialist. 

3c)  There  fs  no  significant  difference  between  actual  and  Ideal  per- 
fcrmance  of  the  school  medfa  specialist  as  perceived  by  principals 
from  rural  schools* 
*3d)  There  Is  no  significant  diff&rence  between  actual  and  ideal  per- 
formance of  the  ^chool  media  specialist  as  perceived  by  principals 
from  urban  schools, 

4a)  There  is  no  significant  difference  between  principals  who  had  and 
had  not  taken  a  formal  media  course(s)  in  their  perceptions  of 
what  should  be  the  actual  role  of  the  school  media  specialist* 

4b)  There  Is  no  significant  difference  between  principals  who  had  and 
had  not  taken  a  formal  nnetiia  course(s)  in  their  perceptions  of 
what  shoLild  be  the  ideal  role  of  the  school  media  specialist. 


*4c)  Thir€  Is  no  si  gniflcant  dif feren ce  beiween  actual  and  ideal 

perfofmance  of  the  sehool  inedla  specialist  as  perceived  by 

principals  vvhp  had  taken  a  metila  course, 
*4d)  There  is  no  si  gniflcant  dif  fireii  ce  between  actual  and  ideal 

perfcmianci  of  the  school  inedia  specialist  as  perceived  by 

principals  %^ho  h&i  not  taken  a  m^dl  a  course. 
*5a)  There  1s  no  si  ^niflcant  d1f feren  ce  between  principals  with 

d1  fferent  tenuTe  in  tNeir  percep'tloTis  of  what  should  be  the 

actual  rple  of  the  schiOQl  media  spedaHstp 
5b)  There  Is  no  si  gnlficaftt  differen  ce  between  principals  with 

di  fferent  t^nu^^e  1n  tlieir  percepilotis  of  what  should  be  the 

ideal  role  pf  ihe  school  media  specialist, 
*5c)  There  Is  no  si gnif leant  difference  hetween  actual  and  Ideal 

performarice  of  the  school  fnedia  spe'ciallst  as  perceived  by 

principal  s  Witti  0-5  years  tenure* 
*5d)  There  Is  no  significant  difference  fcetteen  actual  and  Ideal 

perforrriarice  of  the  school  fnedia  specialist  as  perceived  by 

principal  s  WitPi  600  years  tenure* 
*5€)  There  Is  no  si gni  ficant  difference  between  actual  and  Ideal 

performarice  of  the  schcol  fnedia  specialist      perceived  by 

principal  s  With  1 1-1 5  ymn  tenure. 

Instrmnent  Chosen  fgr  Data  Col  lecting 


The  questl  onnaf  ri  was  selected  to  be  the  type  of  Instrument  used  for 
this  study,    The  qu^st^orinai  re  was  dis^id^d  into  two  parts,  each  having 
its  QVin  purposa,    The  purpose  of  fart ^Qng was  to  obtain  background 

*  HypcDtNesIs  was  rej  ected  at  the  0.05  lewel  of  s  ignif icance. 
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information  on  the  principal  and  the  prlncipaVs  schOoK    This  background 
information  included  conditions  that  may  have  affected  the  principal's 
perceptions  toward  the  actual  and  Ideal  role  of  the  school  media  specialist^ 

The  purpose  of  Part  Two  was  to  determine  the  principal's  perceptions 
of  school  media  specialist's  performance  (both  the  actual  and  the  Ideal 
performance).    Part  Two  was  composed  of  statements  describing  functions 
that  could  be  performed      a  school  media  specialist*    The  principal  read 
each  statement  and  indicated:    a)  his/her  perception  of  the  actual  perfor- 
mance by  the  school  media  specialist  of  the  function  stateds  and  b)  his/ 
her  perception  of  the  level  of  performance  by  the  school  media  specialist 
that  wuld  be  ideal  for  that  function.    The  principal  indicated  his/her 
perceptions  by  giving  each  statement  a  numerical  value.    A  0-5  scale  was 
used  to  measure  the  principal 's  perception  in  regard  to  each  statt^ent. 
The  content  of  part  two  was  based  on  the  recommendations  from  the 
Certification  and  Standards  Committee  of  the  Iowa  Educational  Media 
Association.    Thus,  Part  Two  of  the  quGStionnaire  contained  forty-one 
functions  statements.    These  forty-one  statenients  were  later  (duririg  th^ 
statistical  analysis)  grouped  into  eight  subroles  or  categories*  s^^sterriSs 
administration,  selection,  utilisation,  productionp  leadership,  research, 
and  generaU 

Testing  of  Hypotheses 

When  the  respondents  returned  the  questionnaire^  the  responses  were 
categorized  according  to  their  corresponding  subrole^    For  exaniplet  each 
questionnaire  was  taken  and  the  responses  for  statements  U  17*  18^  and  31 
were  recorded.    These  statements  defined  the  subrole  systems.    These  four 
scores  ware  then  averaged  in  order  to  produce  one  score  for  the  subroU 


ERIC  ^^^^ 


196 


"systems."    Each  questionnairt  produced  sixteen  scorssj  there  were  eight 
subroles  and  two  conditions  (actual  and  Ideal)  under  each  subrole.  The 
data  was  grouped  according  to  this  method  so  that  perceptions  of  subroles 
Instead  of  Individual  functions  could  be  studied.    The  hypotheses  were 
then  tested  by  the  paired  t^- tests  t^-test  or  analysis  of  variance. 
Significance  was  set  at  the  0.05  le^el. 


The  test  results  allowed  for  the  rejection  of  fourteen  of  the  nine- 
teen hypotheses*    On  the  basis  of  the  findings  in  this  study,  the  fonowing 
conclusions  were  made: 

1)  High  school  principals  perceived  a  significant  difference  between 
the  actual  and  ideal  perforniance  of  the  school  media  specialist. 
Significant  differences  were  found  In  all  eight  subrole  areas. 

2)  Each  group  of  principals  perceived  a  significant  dlffirence 
between  the  actual  and  Ideal  performance  of  the  school  media 
specialist.    The  two  groups  that  ware  the  most  satisfied  with  the 
specialist's  performance  were:  a)  those  principals  employed  in 
urban  schools^  and  b)  those  principaTs  employed  eleven  or  more 
years  in  the  school.    The  two  groups  that  were  least  satisfied 
were:    a)  principals  employed  in  small  schooTs  and  b)  principals 
employed  In  ip'ural  schools, 

3)  Principals  from  smatl  and  large  schools  perceived  differently  the 
actual  performance  of  the  school  media  specialist*  Differences 
were  found  in  all  but  one  subrole-utilizatloiu    The  two  groups  of 
principals  perceived  similarly  the  ideal  performance  of  the  school 
media  specialist. 


4)  Principals  from  rural  and  urban  schools  received  different  scores 
in  their  perception  of  the  actual  role  of  the  school  media 
specialist.    The  significant  differences  bitween  the  two  grouos  were 
found  1n  the  subroles:    administration,  selectioni  production, 
leadership,  and  general.    In  all  significant  cases,  principals 

from  urban  schools  produced  the  higher  mean  value.    The  tv^o 
groups  of  principals  did  perceive  similarlj^  the  ideal  role  of  the 
school  media  specialist. 

5)  Princlpais  who  had  taken  a  media  course  and  principals  who  had 
not  taken  a  media  course  did  not  differ  on  their  perceptions  of 
the  actual  role  of  their  school  media  specialist.    Nelthar  did 
they  differ  on  their  perceptions  of  the  ideal  role  of  the  school 
media  specialist, 

6)  There  v^as  a  significant  difference  between  the  perceptions  of 
principals  with  different  tenure  toward  the  actual  roles  of  the 
school  media  specialist.    Principals  employed  16  years  or  more 
produced  a  higher  mean  score  than  principals  who  have  been 
employed  0  to  5  years.    That  is  to  say,  principals  employed  16 
years  or  more  perceived  the  media  specialist  performing  the  sub- 
roles  fiiore  frequently  than  did  principals  employed  0-5  years. 

PHncipaVs  Perceptions  of  Actual  and  Ideal  Roles 

A  ranking  of  actual  and  Ideal  roles  was  produced  from  the  principals 
sampled.    On  the  actual  level,  the  performance  of  the  school  media  specialist 
In  the  various  subrole  categories  ranged  from  perfomis  infrequently  (2.3) 
to  perfornis  sometimes  (3,5)  on  a  5  point  scale.    Utilization  and  selection 
were  the  subroles  being  assumed  most  frequently;  production  and  research 


EKLC 


2i0 


198 


were  the  subroles  perciived  as  being  least  assunted  by  the  school  media 
special ist. 

At  the  ideal  levels  the  desired  perforniance  ranged  from  performs 
sometimes  (3,2)  to  performs  always  (4.2)  for  the  different  subroles. 
Selection  and  utilization  were  the  subroles  that  should  be  assumed  most 
frequently  by  the  school  media  specialist.    Research  and  production  were 
the  subroles  that  should  be  perfonned  least  by  the  school  media  specialist- 

The  subroles  were  ranked  almost  identically  in  both  the  actual  and 
the  ideal  charts-    Principals  were  saying  that  the  order  in  which  the 
subroles  were  being  performed  by  the  school  media  specialist  was  correct. 
The  desire  of  the  principals  was  that  more  time  needed  to  be  spent  on  the 
higher  priority  subroles.    The  only  subrole  that  differed  by  more  than  one 
in  the  rank  ordering  was  the  subrole  general  (which  contained  more 
traditional  functions).    Principals  indicated  that  the  specialists  need 
to  eoncentrate  more  effort  in  the  area  of  administration  and  systems. 

Actual/Ideal  as  Perceived  by  all  Principals 

There  is  a  difference  between  the  actual  and  ideal  performance  of 
the  school  media  specialist  as  perceived  by  principals.    None  of  the 
subroles  were  reported  as  being  performed  at  the  ideal  leveU    It  is  very 
comnon  in  any  field  not  to  have  programs  functioning  at  the  ideal  level. 
Realities  such  as  high  cost,  poor  physical  facilities,  lack  of  pro- 
fessional help!  etc*  may  place  a  barrier  between  the  actual  and  ideal 
perforTnance  levels.    However^  the  value  of  the  results  of  this  study  was 
in  its  showing  the  subroles  that  had  the  greatest  difference  between  the 
two  conditions.    In  other  words  $  the  difference  between  actual  and  ideal 
performance.    The  widest  gap  existed  in  the  administration  subrole.  More 


effort  seems  to  be  needed  from  the  school  media  specialist  in  developing 
and  implementing  media  programs.    The  two  other  subroles  that  needed  the 
most  impt^ovement  were  systems  and  research.    Generally  the  principals 
seem  to  desire  that  a  school  media  specialist  should:    a)  participate  more 
as  a  nieniber  of  the  educational  team;  b)    be  more  involved  with  determining 
goals  for  the  media  programj  and  c)    apply  the  principals  of  research  to 
the  media  program  more  frequently-    Media  specialists  need  to  shift  more 
of  their  efforts  toward  the  learner. 

It  is  this  researcher^s  opinion  that  the  gap  between  actual  and  Ideal 
conditloins  under  the  subrole  of  research  is  due  to  the  fact  that  research 
was  not  a  part  of,  or  not  strongly  emphasized  in  past  media  programs. 

Condi tions  Effecting  Actual  and  Ideal  Perceptions 

Do  certain  conditions  effect  the  principal 's  perception  toward  the 
actual  role  and  ideal  role  of  the  school  media  specialist?  Considering 
perceived  actual  performance,  differences  between  principals  occurred 
for  three  of  the  four  variables-    a)    population  of  the  school,  b)  geo- 
graphical location  of  the  school,  and  c)  tenure  of  the  principal  The 
variable  that  produced  the  greatest  difference  between  principals'  actual 
perceptions  was  the  size  of  the  school.    Seven  out  of  eight  subroles 
differed  significantly  (principals  from  large  schools  producing  the  higher 
score).    It  1s  assumed  by  this  researcher  that  these  results  might  have 
been  due  to  different  budgets ^  facilities,  and  the  number  equality  of 
personnel  working  in  large  school  media  programs. 

No  significant  differences  for  the  ideal  performance  variable  were 
produced  fur  any  of  the  four  conditions  or  variables.    Groups  of 
principals  did  not  differ  in  their  desires  of  how  the  school  media 
specialist  should  perform.    This  indicated  that  there  was  some  unity  within 


the  sample  of  principals.    Discovering  that  principals  desired  simnar 
performance  of  the  school  media  specialist  can  be  valuable  in  program 
planningi  developing  In-service,  educating  future  specialists^  locating 
subroles  where  perhaps  perceptions  need  to  be  changed,  etCp 

Recommendations 

The  data  produced  by  this  study  Indicated  definite  needs  that  should 
be  examined  in  order  to  strengthen  school  media  prograins.    Following  are 
several  recomnendations  towards  fulfilling  these  needs.    Firsts  professional 
media  staffs  need  to  openly  recognize  the  gaps  between  actual  and  ideal 
conditions  In  their  schools.    These  gaps  should  be  clearly  identified  so 
they  may  be  acted  upon.    The  staff  members  then  need  to  discover  what 
conditions  were  causing  the  gaps.    After  the  identification  process  is 
conipleted,  the  staff  then  needs  to  plan  short  and  long  term  activities 
that  will  close  the  gaps. 

The  widest  gaps  discovered  in  this  study  occurred  in  the  areas  of 
administration  in  systemsp  and  in  research.    Since  the  principals 
surveyed  Indicated  a  willingness  for  the  school  media  specialist  to  assume 
greater  responsibility  in  these  areas,  the  school  media  specialist  should 
reassess  their  current  activities  and  prepare  to  provide  more  input  with 
regard  to  these  three  subroles.    Second,  school  principals  should  seek  to 
Improve  communication  within  the  school.    The  principal  should  attempt  to 
encourage  other  staff  members  to  become  aware  of  the  functions  of  the 
school  media  specialist  and  to  use  the  services  of  this  individual. 

Thirds  there  needs  to  be  an  identification  of  the  "new'*  functions  of 
the  school  media  specialist  and  the  "traditional"  functions.  Principals 
ranked  the  traditional  subrole  (general)  slightly  above  three  other  subroles. 


21. 


Coneges  and  universities  that  train  professional  educators  should  make  an 
effort  to  infonn  prospective  school  personnel  about  the  duties  and 
responsibilities  of  the  school  media  specialist  in  order  to  change  the 
Image  of  the  traditional  library  services  and  to  diminish  the  tendency  to 
perpetuate  the  stereotyped  role  of  the  school  media  specialist  (h1stor1cany 
known  as  the  librarian). 

Fourths  there  was  a  substantial  difference  between  the  perceptions 
of  principals  employid  In  larger  scriools  (hiyher  student  enrollment)  and 
principals  employed  in  smaller  schools.    Also,  a  significant  difference 
occurred  between  principals  in  rural  areas  and  urban  areas.    Againi  staff 
members  must  identify  the  reasons  for  the  discrepancies  between  actual  and 
Ideal  conditions  and  strive  to  improve  conditions  for  rural  and  small 
populated  schools.    Perhaps  through  discussion,  in-service  education^ 
pooling  together  outside  resources,  etc.  some  of  the  actual  conditions 
can  come  closer  to  the  ideal.    Curriculum  planners  for  pre-service 
school  media  education  should  provide  programs  that  recognize  and  address 
such  differences  as  those  found  between  rural  and  urban  school Si  large 
and  small  schools* 
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STUDY  OF  ACTUAL  AND  IDEAL  ROLES 
OF  SCHOOL  MiDJA  SPECIALISTS 


The  purpoGO  of    tnis  qut''jrionnalro    if.   to  gftempt   to    Identify  the    Idogl    roltia  and 
functions  alony  with  the  actual    roles  anc   funcfions  of   the  school   media  specialist. 
In  order  to  clarify  th©  turm   'school   media  sp&cialist'^    the  following  definitions 
a  re  p  ro V 1 d ed : 

School  librarian-  prof^ssion&l  responsible  for  print  rriaterlal s  that  are 
housed  in  the  school  media  center  (library)* 

Audio-visual  specialist-  professional  responsible  for  non-print  materials 

snd  equipment  that  are  housed  in  the  achool  media  center  (library)* 

School  media  speelallst-  professional  responsible  for  all  of  the  materials 
(print,  non-print,  equipment)  that  are  housed  in  the 
school  media  center  (library). 

IfttttfNtlNttiiMtt 

Print-  bookSi  magazines,  eza, 

Non-print:  fllmSi  slides,  audio  tapes,  etc. 

tt  li « i»  tt  #  n  «  »  tf  tf 

For  purposes  of   this  suryayi    a  school   meoia   spaeiallst  may   be  considered: 

a)  one  person  responsible   for  ail   of  the  materials   in  The  school  media  center* 

b)  a  two  person  team-    librarian  plus  audio-visual  specialist* 

If  your  school    .mploys  a  two  person  team  as  described  above^   you  will    need  to  think 
of  both  persons  when   rating   the  statements   In  Part  2. 


PLEASE  COMPLETE  THE  SURVEY  BY: 

1.  Checking  the  eorrtct  answer  for  the  questions  asked  in  Part  1, 

2.  Indicating  for  each  statement  your  perception  of  the  actual  condition  in  your  ichool. 
(Part  2i  left  hand  scale) 

3.  Indicating  for  each  statement  your  judnmint  as  to  the  ideal  condition  for  that 
function,  (Part  2,  right  hand  seal i) 

thank  you! 


PART  1 

I,     Check  tht  grade  levels  you  are  responsible  for  as  a  secondary  pflnclpal: 

7-12 
"  8-^12 
"  9-12 
Other 


2.     The  total  student  population  of  your  secondary  school  is: 

Under-250 
251-500 
501-750 
751-1,000 
1  ,001-1,250 
1 ,251-ovef 

3»     The  area  In  which  your  ichool   Is  located  can  be  considired.* 

 Rural  (populations  under-StOOO) 

  City     (populationi  5,001=50,000) 

Urban  (population.'  S0»001-over) 
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4.  How  would  you  describe  your  school  media  CGntcr? 

Sepefatei  one  person  Is  responsibii  for  the  print  mediap  the  othDf 
person  is  responsible  fo^  non-print  media  and  equipment. 
  Unifiedi  one  person  is  responsible  for  all  media, 

5.  The  main  professional  person(s)  in  charge  of  the  school  media  center  is* 

^  ^    ,  ...       '     /   N  (NOTE:  see  definitions  on  first  page) 

School  librarian(s} 

 ^  Audio-visual  spec  1  a  1 i st { s ) 

  School  midia  speclallit(s) 

6.  Have  you  ever  taken  a  formal  media  course  at  a  college  or  univerilty? 
  Yes 

"  No 

****  In  answering  questions  7-9,  count  each  staff  member  only  once. 


7a*     The  number  of  school  librarians  your  school  employs  is: 
 0 

 :i 

2 

Other 


7b.     The  total  work  houri  per  week  for  all  of  the  librarians  are; 

 Under-20  hri. 

,21^39  hrs. 
40-45  hrs . 
hrs. 

 60-79  hrs. 

80-QVtr  hri. 


8a*  The  number  of  audio-visual  specialists  your  school  employs  is: 
Q 

 _1 

2 

Other 

8b.  The  total  work  hours  per  week  for  all  of  the  audio-visual  specialists  hr%\ 

 _Under-20  hrs. 

21-39  hrs. 

4Q-45  hrs. 

46-59  hrs. 
 60-79  hrs. 

80-Qver  hrs* 


9a*  The  number  of  school  media  specialists  your  school  employs  is: 
Q 

 ^1 

2 

Other 

9b.  The  total  work  hours  per  week  for  all  of  the  school  media  ipeciallsts  are: 

 Under-20  hrs. 

^2U39  hrs. 
40-45  hrs. 
46-59  hrs. 
60-79  hrs. 
80-ovir  hri. 


10.  The  number  of  years  that  you  have  been  employed  as  a  secondary  school  principal 
at  your  school  Is: 

 1-5  yrs. 

_6-10  yrs. 
1 1-15  yrs. 
IS-'Qver  yrs. 


ACTUAL 
PERFORHANCE 


lOfAL 
PERFORIiANCE 


^ 

z 

^ 

m 

UJ 

m 

a. 

Of 

O 

O 

u> 

C£ 

a£ 

UJ 

UJ 

UJ 

a. 

£L 

0 

1 

2 

3 

4 

s 

m  y%  ^ 

yj  UJ 

^  m  m 

Lii  t/^ 

UJ  Si  >■ 

^    1^  UJ  or  ^ 

^     L4,  ^  UJ 

o  ^  o  ^  ^ 

3      U_  !^  Uk.  ^ 

o  m 

^    UJ  yl 

^    ^    Z  ^  ^  £ 

^  ^  ^ 

H-  ^  o  o  o  a 

UJ    u^  u^  u_ 

s    >  a  ^  ^ 

o    UJ    UJ  UJ  u, 

Q  ^  Q»  ^  a,  & 


22.  Participates  in  the  planningi  arrangement,  and  utili-     0   12  3    4  5 
zation  and  development  of  media  facilities  which  will 
support  the  objectives  of  the  media  program  and  the 
instructional  program* 

5 
5 


0 

1 

2 

3 

4 

5 

23. 

Operatei  production  equipment. 

0 

1 

2 

3 

4 

0 

1 

2 

3 

4 

5 

24, 

Evaluates  and  selects  production  equipmint  and  supplies 

-0 

1 

2 

3 

4 

0 

1 

2 

3 

4 

5 

25. 

Assists  teachers  and  students  in  identifying,  obtainingiO 
and  adapting  media  to  meit  their  needs. 

1 

2 

3 

4 

0 

1 

2 

3 

4 

5 

26, 

Providei  guidance  in  reading,  liitening,  and  viewing 
experiences  for  students  and  teachers. 

0 

1 

2 

3 

4 

0 

I 

2 

3 

4 

5 

27. 

Keeps  school  media  center  quiet. 

0 

1 

2 

3 

4 

0 

1 

2 

3 

4 

5 

28. 

Keeps  an  adequate  equipment  inventory. 

0 

1 

2 

3 

4 

0 

1 

2 

3 

4 

5 

29, 

Retrieves,  interprets,  and  evaluates  available  research 
related  to  the  educational  program. 

0 

1 

2 

3 

4 

0 

1 

2 

3 

4 

5 

30. 

Providei  and  protects  the  right  of  access  to  informa- 

0 

1 

2 

3 

4 

tion  by. faculty  and  students  within  the  tt^isting 
1  egal  f  rariewprk  . 

0    12    3  4    5     31.  Determines  goals  for  the  media  program  as  an  integral      0  1    2  3    4  5 

part  of  the  educational  program  of  the  schoor  and 
d1 strict. 

0    12    3  4    5     32.  Establishes  procedures  for  effective  and  efficient  0  1    2  3    4  5 

acquisition,  processing,  cataloging,  circulation,  dis- 
tribution, and  maintalnance  uf  materials. 

0    12    3  4    5     33,  Establishes  job  specifications  and  applies  principles      0  12    3  4 

of  personritl  management. 

0    1  2    3  4    5     34.  Develops  and  applies  criteria  for  evaluating  and  0  12    3  4 

selecting  a  variety  of  materials  and  oquipment. 

0    12    3  4    5     35,  istablllhis  and  administers  processes  and  procedures        0  12    3  4 

for  previev^,  evaluation,  selection,  and  withdrawal 
of  materials  and  equipment. 

0    12    3  4    5     36,   Instructs  and  supervises  others  In  media  design  and         0  12    3  4 

producti  on . 

0    12    3  4    5     37.  Designs  and  adapts  an  Identified  research  study  for  the  0  1    2    3  4 

development  and  advancement  of  the  media  program. 

0    12    3  4    5     38,  Delivers  needed  Qquipment  to  the  classrooms.  0  12      3  4 

0    12    3  4    5     39,  Teaches  skills  In  the  retrieval  and  utiliiatlon  of  0  12      3  4 

materials  and  equipment  to  students  and  teachers. 

0    12    3  4    5     40.  Designs  production  facilities  and  establishes  basic         0  12      3  4 

routines  for  the  operation  of  those  facilities, 

0    12    3  4    5     41,  Designs,  develops,  and  writes,  proposals  for  the  acqui-  0  12      3  4 

sition  of  local,  itate,  and  federal  funds  to  support 
and  expand  media  progfams* 

In  oH.di.K  to  Idtntliy  thok^  6a^\/^y&  that  fiaue  been  ^e^u^necf,  PLEASE  ^tignt 

VOUn  NAMEt    ________ 


THANK  YOU!  o  .  /^'^^^  SCHOOL  t 


PART  2 

P4efl^ft  the.  number  yu'hich  faeif  h^p%Q^iir.ti  ijau^  piinc^pt^on  oi  the.  attual  and  <dQai 

ACTUAL  IDEAL 


^  ^  ^ 


O'  yj  yi 

yJ  a  3« 

S  tkj  cr  ^ 

3    M-  S  uj  3 

^     Q     ^  O  ^ 

%      LU  ^ 

^      LU      t/1  LO  to 

*^  ^  i  i  i  a 

^   ©  o  o  o 


a 


^  ^ 
O  Uj  UJ 
Q    a  ^ 


yj  1-*^ 


5f  Q 


4/5  to 


to 


a 


0 

1 

2 

3 

4 

5 

L 
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Abstract 

The  purpose  of  this  study  was  to  investigate  instructional  method-- 
ology  related  to  the  effectiveness  of  different  Instructional  feedback 
strategies  that  can  be  used  in  regular  classroom  situations*  Experimental 
conditions  were  created  to  investigate  v^hetherj   in  addition  to  mastery 
learning  strategi'es,   (a)  student  se  1  f  ^eva  1  uat  ion  of  their  products  using 
specified  criteria  or  (b)  student  sel f ^eval uat ion  and  instructor  feed- 
back on  the  accuracy  of  their  evaluations  would  affect  either  the  student^s 
ability  to  use  stated  criteria  to  Judge  their  own  performances ^  or  would 
actually  improve  student  performance*     Fifty-six  undergraduate  students 
were  assigned  to  either  the  control  group,  the  self-evaluation  group, 
or  the  sal f-evaluat ion/feedback  group  to  measure  the  effects  of  these 
var  iables , 

Results  indicate  that  the  association  between  student  self-evaluations 
and  instructor  evaluations  on  a  final  class  project  was  negatively  affected 
by  student  se 1 f -eva 1 uat ion  only,  and  positively  affected  by  the  combfna^ 
tion  of  self-evaluation  and  instructor  feedback  on  the  accuracy  of  the 
student  self-evaluations.     Feedback  was  shown  tr  be  most  effective  in 
situations  in  which  students  had  too  much  confidence  in  the  quality  of 
thei  r  materials. 

These  findings  indicate  that  the  combination  of  student  self^ 
evaluation  and  instructor  feedback  ^jn  the  accuracy  of  those  evaluations 
does  affect  the  ability  of  students  to  use  prespeciffed  criteria  to 
evaluate  their  own  work  during  the  developmental  process.     In  addition. 
It  positively  affects  the  performanGe  of  students  who  overestimate  the 
quality  of  their  work. 
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In  an  effort  to  increase  the  effectiveness  of  Instuction,  many  universi- 
tias  advocate  the  use  of  ■'objective  based"  i nstruct lonal  programs*  In  courses 
in  which  objectives  are  used,  students  are  told  what  performances  are  expected 
of  them  during  the  cQurse  and  are  often  given  the  criteria  by  which  these  per^ 
formancas  will  be  judged.  It  Is  assumed  they  will  use  the  criteria  given  to 
evaluate  the  quality  of  their  own  performances  and  to  alter  performances  which 
are  not  as  they  should  be. 

When  mastery  learning  Is  the  Issues  the  ability  to  self-evaluate  takes  on  In^ 
creased  importance.     If  learners  are  unable  to  evaluate  their  own  work,  if 
they  are  unable  to  discriminate  between  correct  and  incorrect  performances  during 
the  learning  processs  the  results  of  instruction  will  be  less  than  optimal.  The 
question  to  be  considered  is  v/hether  students  are  able  to  use  objectives  and 
criteria  to  evaluate  their  own  performances,  and  If  so,  whether  the  ability  to 
sel f ^eva I uate  improves  with  practice. 

Experimental  situations  were  created  to  investigate  the  effects  of  two  in- 
dependent variables^     (a)  learners-  sel f^evaluatlon  of  their  products  using 
specified  criteria  and  (b)  learners*  sel f =eva 1 uat ion  with  Instructor  feedback 
of  the  accuracy  of  their  evaluationsi  on  two  dependent  variables:   (a)  learners* 
ability  to  use  stated  criteria  for  Judging  their  own  products,  and  (b)  learners' 
performances  on  course  objectives* 

Review  of  Literature 

Literature  on  systematically  designed  Instruction  and  the  variety  of 
instructional  strategies  that  can  be  employed  to  improve  learning  Is  abundant. 
For  the  purposes  of  this  study,  literature  that  contained  descriptions  of 
(a)  systematical ly  des igned  instructional  procedures  related  to  object  Ives, 
standards,  and  practice  and  feedback  and  (b)  sel f -^eva  luat  ion  as  an  Instruc- 
tional strategy  was  reviewed, 
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I n s  t  rue  t  i  ona 1  s  t  ra  teg  i  e  s .    The  writers  first  eKplored  literature  on 
systematic  instructional  design  procedures  and  the  effects  of  using  system- 
atically designed  instruction.    Particular  areas  studied  were  performance 
objectives,  performance  standardss  and  practice  and  feedback. 

A  number  of  learning  theorists  state  that  learning  will   Improve  as  a 
result  of  pre«5peci f led  objectives  (Gagnl  &  Briggs,  197^8  Gerlach  &  Ely,  in 
press;  Kibler,  Barker  £  Miles,  1970;  Klbler  6  Basset,  1977i  Mager,  igSl). 
More  research  is  needed  on  the  development  of  strategies  that  will  increase 
the  ability  of  students  themselves  to  use  objectives  In  learning  situations* 
One  question  about  objectives  which  remains  unanswered  is  whether  students 
can  effectively  use  only  statements  of  objectives  for  maximum  instructional 
effectiveness  and,   if  notj  what  other  instructional  techniques  preceeding 
instruction  can  be  employed  along  with  objectives  to  facilitate  learning. 

The  specification  of  performance  standards  is  widely  accepted  as  an 
integral  part  of  an  instructional  objective.    When  performancs  evaluation 
is  judgmentaU  when  there  is  no  clearcut  right  or  wrong  performance,  the 
specification  of  performance  criteria  becomes  appropriate.    When  a  simple 
right  or  wrong  response  is  not  possible,  Dick  and  Carey  (1978)  suggest  that 
instructional  objectives  should  include  a  checklist  of  the  types  of  behaviors 
which  will  be  expected  when  the  performance  of  students   Is  Judged,  This 
should  give  students  a  clear  understanding  of  the  nature  of  the  required 
performance.     However,  the  degree  to  which  learners  are  able  to  use  criteria 
in  shaping  their  behaviors  has  not  yet  been  established  (Carey,  1976). 

The  use  of  practice  and  feedback  are  universally  accepted  as  important 
aspects  of  instruction.    Both  are  regularly  included  In  systematically 
de&igned  instructional  materials.    Research  on  practice  and  feedback  were 


reviewed  in  this  study  relative  to  sel f^evaluation  or  student  evaluatfon 

of  their  own  work.    Kulhavy,  Yekovich  and  Dyer  (1976)  have  shown  that*  in 

progrannmed  Instruction,  certain  conditions  maximize  the  effects  of  Instructor 

feedback  on  learner  performance.    They  found  that  feedback  in  programmed 

instruction  is  most  effective  when  a  learner  believes  a  test  response  is 

correct  when,  in  fact,  it  is  incorrect.    They  say  that,  in  this  case,  learner 

response  confidence  is  Ineorrectly  high.    When  a  learner  believes  a  response 

is  incorrects  and  It  is,  Kulhavy  et  al  *  state  that  feedback  is  not  effective 

because  learners  do  not  understand  either  the  subject  matter  being  tested, 

the  test  quest  Ion  J  or  both. 

in  programmed  instruction,  feedback  indicates  to  the  learner  whether  a 

particular  response  to  a  program  frame  Is  correct  or  incorrect ^  and  the 

effects  of  reedback  are  measured  by  a  subsequent  test  covering  the  same 

material .     In  some  instructional  situations,  the  feedback  given  on  one 

performance  is  expected  to  help  the  learner  with  future  performances  which 

are  similar.    When  complex,  mul t 1 ^d i mens i onal  performances  are  given, 

learners-  confidence  in  this  performance  can  be  measured  by  having  them 

evaluate  each  aspect  of  their  own  performance,     Se  1  f --eval  uat i on  becomes  an 
indicator  of  learner  response  confidence.     If  instructor  feedback  Is  then 

given  on  the  quality  of  learner  sel  f  ^eval  gat  i  ons  ,  In  addition  to  the  perform^' 

ance  of  tasks  to  be  learned,  a  double  feedback  condition  exists.    The  effects 

of  feedback  on  student  performance  should  be  noticed  under  these  conditions 

regardless  of  learner  confidence*  especially  when  the  feedback  given  on 

performance  and  on  sel f ^eval uat Ions  is  instructional  and  causes  students 

to  attend  more  to  objectives  and  standards  for  acceptable  performance. 


Sel f-evaluat ion .    One  method  for  determining  the  degree  to  which 
learners  are  able  to  use  objectives  and  criteria  statemants  to  shape  their 
performances  is  to  ask  learners  to  use  them  to  evaluate  theJr  own  work* 
Those  who  are  able  to  evaluate  their  work  using  pre-speci  f  i  ed  criteria 
probably  possess  a  clear  understanding  of  the  objective  and  criteria  concerned. 
On  the  other  hand,  learners  who  are  unable  to  use  performance  criteria  to 
judge  the  quality  of  their  work  after  its  completion  are  probably  unable 
to  use  the  same  criteria  during  its  production.    Learners  who  are  unable  to 
use  criteria  during  performances  would  not  be  expected  to  perform  as  well 
as  those  who  have  the  ability  to  apply  statements  of  criteria  when 
developing  instructional  products. 

Clark  (1938)  found  that  students  were  able  to  evaluate  their  own  perform-' 
anee  on  college  level  algebra,  quantitative  analysis,  and  chemistry  problems. 
The  correlation  between  student  scores  and  Instructor  scores  was  .80.  Bennent 
(1958)  supports  the  argument  that  students  are  able  to  evaluate  their  ov^n 
performances , 

Both  Clark  and  Bennent  report  that  students  g^ve  themselves  the  same 
grade  as  their  instructors  in  a  high  percentage  of  cases.    Estimating  a  single 
grade,  or  single  number  on  a  S^polnt  scale  seems  to  be  an  easier  task  than 
determining  how  closely  several  parts  of  a  compteK  performance  match  the 
related  performance  criteria*    When  one  performs  in  the  fields  grades  are 
not  a  consideration,  Job  performance  is,  and  evaluations  are  based  on  how 
elosely  the  actual  performance  approximates  the  eKpected  performance. 

The  question  of  whether  sel  f -=eva  1  uat  Ion  actually  promotes  learning 
remains  unanswered,    McEowen  (1957)  found  that  learning  was  unaffected  by 
students'  evaluation  of  their  materials.     In  this  study,  however,  performance 
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had  no  effect  on  course  grades,  so  students  had  Httle  at  stake.    Had  the 
performance  affected  the  grades,  student  Interest  in  learning  might  have 
be©n  greater  and  the  results  might  well  have  been  different. 

In  a  study  involving  sculpturej  graphics,  paintlngj  and  drawing  skills. 
Fried  (1965)  found  that  although  sculpture  improved  as  a  result  of  self- 
evaluationi  the  other  skills  studied  did  not.    Noting  the  I nconcl us i veness 
of  his  own  finding.  Fried  concluded  that  the  value  of  sel f -evaluat Ion  was 
still  undetermined,  and  recofnmended  continued  research  In  the  area. 

In  a  study  conducted  in  two  Air  Force  technical  schools ^  Duel  (1958) 
found  that  achievement  was  improved  when  students  were  given  formal  and 
periodic  opportunities  to  evaluate  their  own  work,    A  study  in  which 
sel f ^eva 1 uat i on  had  a  positive  effect  on  students'  mechanical  drawing  skills 
was  ronducted  by  Irwin  (1973)'    He  found  that  students  who  evaluated  their 
own  materials  throughout  the  school  year  learned  more  as  measured  on  a 
standardized  test  which  assessed  mechanical  drawing  skills.     In  this  study, 
however,  results  were  confounded  because  the  self-evaluating  group  had  a 
significantly  higher  mean  IQ  than  the  control  group. 

Research  into  the  effects  of  sel f ^eva luat i on  has  generally  been  concerned 
With  either  student  ability  to  sel f -eval uate,  or  the  effect  of  sel f^evaluat i on 
on  student  performance.    An  indication  from  the  literature  is  that,   in  certain 
casaSj  students  are  able  to  evaluate  their  own  rnaterlals,  although  the  evidence 
IS  far  from  conclusive.    Evidence  has  been  cited  which  both  supports  and 
contradicts  the  assert  ion  that  sel f ^eval uation  improves  student  performance. 
Little  has  been  written  on  the  effects  of  (a)  student  practice  on  evaluating 
their  own  work  and  of  (b)   instructor  feedback  on  student  sel  f -^evaluation. 
The  combination  of  systematic  instructional  procedures  with  sel f ^eva luat i on 
as  an  additional  feedback  strategy  may  prove  to  make  instruction  more  effective* 
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Hypotheses 

The  literature  concerning  learner  self-evaluation,  though  limfteds 
does  indicate  that  students  may  possess  at  least  a  limited  ability  to 
evaluate  their  own  work  and  that  periodic  self-evaluation  may  improve  learner 
performance . 

The  specific  hypotheses  tested  in  this  research  are  the  following' 

1,  The  correlation  between  learner  and  instructor  evaluation  scores 
on  a  final  course  project  will  be  higher  when  instructor  feedback  ib  given 
on  previous  learner  self-evaluations  throughout  the  term  than  when  no 
instructor  feedback  on  self-evaluation  is  given,  or  when  students  do  not 
participate  in  sel f^evaluat ion  throughout  the  term. 

2,  Learner  performance  on  a  final  class  project  will  be  better  when 
students  evaluate  their  own  materials  throughout  the  term  and  receive 
Instructor  feedback  on  their  self ^evaluations  than  when  students  do  not 
receive  feedback  on  their  self -evaluationSs  or  do  not  participate  in 

se 1 f ^evaluation  throughout  the  term. 

3,  Learner  performance  on  a  final  product  will  be  better  when 
feedback  Is  given  on  the  quality  of  the  students'  evaluation  of  their  own 
work  than  when  instructor  feedback  is  given  only  on  the  quality  of  the 
product  for  a  subgroup  of  students  who  incorrectly  assume  that  their  initial 
products  are  well  produced. 

Learner  performance  on  a  final  product  will  be  better  when  feedback 
is  given  on  the  qualfty  of  the  Initial  product  and  on  student  evaluations 
of  the  project,  than  when  feedback  is  given  only  on  the  product  for  a  subgroup 
of  students  who  correctly  assume  their  product  Is  not  well  produced. 


PI  lot  Study 

In  an  attempt  to  identify  those  experimental  procedures  which  could 
most  easily  be  adapted  to  an  ongaing  instructional  progranrij  a  pilot  study 
was  conducted  during  the  spring  semester  of  1978  at  Arizona  State  Universityi 
Tempes  Arizona, 

Hethod 

Sample .    Three  instructors,  and  n^n©  undergraduate  media  production 
classes  participated  in  the  study*    Three  intact  classes  were  assigned  to 
each  of  the  three  treatment  groups;  control ,  sel  f ^^eval  uat i  on ,  and  feedback. 
Each  instructor  was  responsible  for  only  one  treatment  group. 

Procedures .    The  control  group  used  performance  objectives j  criteria 
statements^  and  instructor  feedback  to  produce  a  mediated  unit  of  instruction. 
There  v/ere  three  major  checkpoints  during  the  semester  at  which  students 
formally  submitted  their  materials  for  instructor  evaluation  and  feedback* 
The  self-evaluation  group  followed  procedures  previously  described  for  the 
control  group.     In  addltionj  prior  to  submitting  their  materials  for  instructor 
evaluation^  they  used  the  instructor  evaluation  form  to  assess  the  qualfty  of 
their  own  work.     Instructor  feedback  on  the  quality  of  the  materials  was  the 
same  for  both  the  control  and  the  sel f^eval uation  grQups  ,    The  third  or 
feedback  group  produced  their  materials  In  the  same  manner  as  the  other  two 
groups.     Like  the  sel  f --eva  1  uat  i  on  group,  prior  to  submlttihg  their  fnaterlals, 
they  evaluated  their  own  work  and  submitted  the  evaluation  forms  to  their 
instructors*     Instructor  feedback  for  this  group  consisted  of  both  feedback 
on  the  quality  of  the  products  and  on  the  quality  of  the  student  evaluations. 

To  enable  comparisons  in  sal f -eval uat i ons  among  the  three  groups ^  the 
control  group  used  the  evaluation  forms  to  .assess  their  final  products  at 
the  end  of  the  semester, 
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Results ,    Pearson  product  moment  correlations  were  used  to  compare  the 
evaluation  scores  of  students  with  those  of  their  instructors  in  the  control, 
self-evaluat ion,  and  feedback  groups  on  the  final  product.  Significant 
correlations  (p  ^  ,05)  were  observed  between  student  and  instructor  ratings 
in  both  the  control  and  the  feedback  groups,    A  slightly  negative,  but  non- 
significant  correlation  was  observed  in  the  sel f^eval uation  group. 

A  one-way  analysis  of  variance  imong  group  5  revealed  no  significant 
differences  (p  >^  ,05)  among  the  performance  scores  of  the  three  groups. 

Discussion 

As  a  result  of  the  pilot  study j  a  number  of  procedural  problems  were 
identified.     Due  to  the  nature  of  the  procedural  problems,  data  collected 
during  this  study  is  not  valids  and  it  Is  not  general  i zable  to  other  instruc'^ 
tlonal  settings.    Changes  in  procedures  which  were  identified  as  being 
problematic  are  described  in  the  following  paragraphs. 

The  first  procedure  which  was  found  to  be  unacceptable  was  the  assignment 
of  one  treatment  to  each  instructor-     Training  sessions  were  held  to  establish 
scoring  guidelines^  and  a  high  degree  of  inter-rater  rel iabi 1 i ty  was  established 
However*  during  the  study,  which  spanned  an  entire  semester,  a  distinct  pattern 
of  instructor  bias  developed*     One  instructor  seemed  to  be  more  lenient  in 
grading  than  did  the  others,  and  also  offered  students  more  assistance  in  the 
development  of  their  products.    Students  were  assured  of  high  grades  by  the 
instructor  prior  to  formal   instructor  evaluation.    As  a  result,  students  knew 
their  scores  before  submitting  their  products,  and  the  relationship  between 
student  and  instructor  scores  was  no  doubt  inflated  In  this  case.     It  was 
determined  that  if  each  instructor  delivered  all  three  treatments,  then  the 
effects  of  this  type  of  instructor  bias  would  be  minimized. 

A  second  problem  which  was  encountered  concerned  the  instruments  used 
for  both  student  and  instructor  evaluations.    The  objectives  list  and 
grading  criteria  were  on  separate  forms  and  although  students  had  both 


forms  in  their  possession  at  the  time  of  self-avaluation *  they  rarely 
referred  to  the  criteria  sheet  when  grading  their  products*    The  objectives 
and  criteria  were  combined  into  one  form  to  help  students  focus  on  the 
criteria  when  assessing  the  quality  of  their  materials. 

The  pilot  study  helped  refine  experimental  procedures  and  enabled  the 
researchers  to  conduct  the  main  study  in  an  ongoing  Instructional  settfng- 
With  the  exception  of  instructor  treatment  assignment  and  evaluation 
Instrument  design^  the  procedures  used  in  the  pilot  study  were  also  used  In 
the  main  study. 

Method 

Sample 

The  study  was  conducted  during  the  1978  summer  session  at  Arizona  State 
University,  Tempe,  Arizona.    Student  participants  were  enrolled  in  IME  411, 
Audiovisual  Materials  and  Procedures  in  Education.    A  total  of  56  preservice 
and  Inservice  teachers  in  three  class  sections  participated.    Students  In 
each  section  were  blocked  according  to  three  levels  of  grade  point  average; 
low,  average,  and  high,  and  then  randomly  assigned  to  either  the  control^ 
first,  or  second  treatment  group.    At  the  onset  of  the  study,  there  were  20 
students  In  the  control  group,  16  in  the  first  treatment  group,  and  20  In  the 
second  treatment  group.     During  the  conduct  of  the  study,  eight  students 
were  dropped  from  the  control  group  because  their  instructor  confused  their 
treatment.    Each  course  instructor  taught  the  control  and  the  two  treatment 
groups  In  order  to  avoid  instructor  bias. 

Mastery  Instructional  strategies  Including  use  of  performance  objectives, 
specified  criteria,  examples  of  acceptable  products,  relevant  practice,  and 
Instructor  Feedback  on  the  quality  of  students'  products  were  the  instructional 


techniques  used  in  all  three  treatment  groups  to  help  students  produce  one 
mediated  and  one  non-mediated  unit  of  instruction.    The  instruction,  practice 
and  feedback,  and  assessment  procedures  occurred  throughout  the  course.  There 
were  three  major  checkpoints  at  which  students  formally  submitted  their 
products  for  instructor  evaluation  and  feedback. 

The  control  group  followed  the  Instructional  procedures  outlined 
previously  to  produce  their  materials  during  the  course*    Students  in 
the  first  treatment  group  followed  these  same  proceduresj  and 
in  addition*  prior  to  submitting  their  materials  for  Instructor  evaluation  at 
each  checkpoints  students  in  this  group  used  the  instructor  evaluation  form 
to  evaluate  their  materials.    The  self-evaluations  were  submitted  with  the 
materials  and  were  not  seen  again  by  students.    The  evaluation  form  included 
a  listing  of  all  aspects  of  the  products  to  be  evaluated  and  the  specific 
criteria  to  be  used  itj  the  evaluation,     (Sample  instructor  eviluatfon  forms 
can  be  seen  in  AppendiK  A,)    This  treatment  group  is  referred  to  as  the 
sel f^evaluat ion  group* 

The  second  treatment  group,  referred  to  as  the  feedback  group,  followed 
the  same  procedures  as  the  self-evaluation  group  with  the  addition  of  one 
procedure.    When  instructor  feedback  was  given  on  the  quality  of  student 
materials,  students  were  also  given  feedback  on  their  self-evaluations* 
Instructors  asked  students  why  they  graded  themselves  the  way  they  did, 
based  on  the  criteriaj  and  showed  students  how  their  evaluations  compared 
with  instructor  evaluations. 

Students  In  both  the  control  and  se  1  f -^evaluat ion  groups  received  one 
type  of  feedback  feedback  on  the  quality  of  their  materials.  Students 
In  the  feedback  group  received  feedback  on  the  quality  of  their  materials 
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and  also  on  their  self-evaluations.     It  Is  important  to  note  that  the 
InstruGtors  did  not  see  the  student  sel f ^evaluations  until  after  student 
products  were  graded.    This  eliminated  Instructor  bias  caused  by  student 
perceptions  of  the  quality  of  their  work. 

Performance  Standards 

Each  aspect  of  student  products  was  graded  by  course  instructors  on  a 
scale  of  1  to  10*  with  10  being  the  highest  score  possible.     Grading  criteria 
were  established  by  course  instructors  prior  to  the  beginning  of  the  study* 
Instructor  scores  were  used  as  tlhe  standard  against  which  all  comparisons 
were  made* 

Pes  ign 

In  this  study  a  posttest^only  control  group  design  was  used  to  measure 
the  effects  of  student  sel f -^evaluat Ion  of  instructional  products,  and 
instructor  feedback  on  the  quality  of  sel f^evaluation  on  student  performance 
on  the  final  class  product*    An  analysis  of  variance  at  a  significance  level 
of  *05  was  used  to  measure  the  effects  of  these  two  independent  variables, 
A  Pearson  product  moment  correlation  at  a  significance  level  of  ,05  was 
used  to  measure  the  relationship  between  student  evaluations  and  instructor 
evaluations  of  all  instructional  products* 

Resul ts 

Self-evaluation  and  feedback  on  the  accuracy  of  self-evaluation  were 
studied  to  ascertain  their  effects  on  (a)  student^s  evaluation  of  their 
own  performance  and  (b)  on  student  achievement. 


Effects  of  Self-Evaluation  and  Feedback 

on  the  Relationship  Between  Student 
and  I  nit  rue tor  bcores\ 

It  was  hypothesized  In  this  study  that  the  correlation  between  student 
evaluation  scores  and  Instructor  evaluation  scores  on  the  final  course  project 
would  be  higher  when  instructor  feedback  on  student  evaluations  of  other 
class  projects  was  given  than  when  no  Instructor  feedback  was  given,  or  when 
no  student  evaluation  took  place* 

Student  and  Instructor  evaluation  means  and  standard  deviations  for  the 
final  course  project  and  the  correlation  between  them  are  shown  In  Table  1* 
For  the  contro!  group,  the  student  evaluatlan  mean  was  57*9  (maKlmum 
score  ^  60)  and  the  Instructor  mean  was  56.82^  yielding  a  difference  of  1,08 
points  with  the  correlation  between  them  being  *68  (p  >^  .05).    For  the  self- 
evaluation  groupj  the  student  mean  was  58. 5^  with  the  instructors  giving  a 
tnewn  ^wre-^F  SgT^^rd-^fPerm  between  these 

two  scores  was  -*07i  which  was  not  significant.    Students  in  the  feedback 
group  predicted  a  mean  score  of  55.70  while  the  instructors'  mean  equaled 
5^*75f  yielding  a  difference  of  ,S5  points*    The  correlation  between  these 
scores  was  ,81   (p  >^  .05)* 


Insert  Table  1  about  here 

Figure  1  contains  an  illustration  of  the  correlations  between  student 
and  instructor  scores  at  the  first  and  third  check  points  for  the  self-' 
evaluation  and  feedback  groups.    There  was  very  little  change  in  student 
ability  to  evaluate  their  own  work  In  the  sel f ^eval uat Ion  group.    At  the 
first  checkpoint  J  the  correlation  between  student  and  instructor  scores  was 
-.048  and  at  the  third  checkpoint,  -,076,    Neither  of  these  correlations  was 
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Table  1 


Student  and  Instructor  Mean  Performance  Scores  and  Standard  Deviationi  on 
'inal  Class  Project  and  Correlation  between  the  Two  Means  for  Each  Treatment  Group 


Group 


Student  n" 


I  nstructor 


Standard 
Dev lat  ion 


Cor  re  1 at  ion 


Control 


3-65 


Self-evaluation         58.50  (U) 


53.67 


5-57 


-.07 


Feedback 


55.70  (20) 


5^.75 


8.26 


.81** 


Note.    Maximum  score  =  60. 

"'Numbers  in  parenthesis  indicate  the  number  of  studefits  in  each  group. 


significant.    There  was  little  association  at  the  outset  of  the  study  and 
only  slightly  more  at  the  conclusion. 

On  the  other  hand^  the  correlation  between  student  and  instructor  scores 
for  the  feedback  group  was  ,^9  (p  <  .05)  at  the  first  evaluation  checkpoint 
and  .81  .(p  <  *05)  at  the  third.    The  relationship  between  the  two  sets  of  scores 
for  this  group  increased  approximatel y  65  per  cent. 

Insert  Figure  1  about  here 


Effects  on  Performance 

It  was  predicted  that  students'  performance  on  the  final  class  project 
would  be  better  when  students  evaluateo  previous  class  products  and  received 
instructor  feedback  on  the  self-evaluations  than  when  they  completed  self- 
evaiuatlons  alone  or  did  not  cDmplete  self-evaluations  at  all.    A  one  way 
analysis  of  variance  revealed  no  significant  difference  between  group  means* 

To  measure  the  relationihip  between  student  confidence  in  their  materials 
as  measured  by  their  self-evaluation  scores  at  the  first  evaluation  checkpoint, 
and  the  effect  of  instructor  feedback  on  student  self-evaluations,  students 
were  blocked  into  two  groupsi     (a)  student  evaluations  above  50  indicating  a 
high  degree  of  Gonfidence  in  their  materials  and  instructor  evaluations  below 
50  on  the  same  materials,  and  (b)  both  student  and  Instructor  evaluations 
below  50  indicating  low  student  confidence  in  their  materials.    Table  2 
shows  the  means  and  standard  deviations  for  these  two  subgroups  for  both 
the  sel f^eval uat ion  and  feedback  groups. 


Insert  Table  2  about  here 
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Figure 

Student  and  Instructor 
on  the  Fi  rst  and 


1 

Eval uat Ion  Correlati ons 
Fi nal  Evaluations 


Table  2 

Instructor  Mean  P&rformmcm  Scores  and  Standard  Deviations  on  Final  Project 
Blocked  on  Student  Confidence  In  Their  First  Project 


Mean  Standard 
Group  Score  n^  Deviation 


Incorrect  high  confidence  ^ 

Self-evaluation  53*09  (10)  5.65 

Feedback  58.25  (l2)  1.82 

Correct  low  confidence 

55^25  thy  5.85 

Self-eVal uation 

49.50  (  8)  11 ,29 

Feedback 


^Numbers  in  parenthesis  Indicate  the  number  of  students  In  each  group. 


Note .    Maximum  score  -  60. 
^Numbers  in  parenthesis  Indicate  the  numb 


A  one'-v/ay  analysis  of  variance  indicated  that  when  student  confidence 
was  tncorrectly  high,  there  was  a  significant  difference  between  the  self^ 
evaluation  and  feedback  groups  £^  9.03  (1,  21)  p<  .01.    When  student 
confidence  in  materials  was  correctly  low,  the  feedback  group  mean  was 
approximately  sik  points  lower  than  the  mean  score  of  the  self-evaluation 
group. 

Di  scuss ion 


a 


The  purpose  of  this  study  was  to  Investigate  the  effects  of  (a)  student 
self-evaluation  and  (b)  instructor  feedback  on  the  self-evaluations  on  the 
abiUty  of  students  to  evaluate  their  own  materials  and  on  their  performance. 
Although  criteria  statements  are  often  included  In  performance  objectives, 
there  has  been  little  evidence  which  indicates  that  students  can  always  use 
criteria  to  shape  their  performance  (Carey.  76).    The  instruction  used  in 
this  study  Included  objectives  with  criteria  statements  and  relevant  practice 

nd  feedback.  It  conformed  to  the  prescription  for  well  developed  instruction 
as  specified  by  Gagnl  and  Briggs  (197^).  Dick  and  Carey  (1978),  and  Ely  and 

Gerlach  (In  press) . 

The  relationship  between  student  score  predictions  and  instructor  scores 
on  the  final  class  product  was  reasonably  high  for  the  control  group, 
r  =  .68  (p         .05),  supporting  the  work  of  Clark  (1938)  and  Bennent  (1969). 
However,  the  combination  of  sel f=evaluation  and  Instructor  feedback  on  se1f= 
evaluation  raised  this  figure  to  .8T.  indicating  that  the  skill  of  self- 
evaluation  can  be  developed.    These  findings  support  the  hypotheses  stated 

in  this  paper.     Criteria  specific  feedback  on  self-evaluation  seems  to  be  an 
effective  way  to  develop  student  skills  in  estimating  the  quality  of  their 

performances* 
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The  correlation  between  student  and  instructor  evaluation  scores  for  the 
control  group  was  much  higher  at  the  third  evaluation  checkpoint  than  the 
correlation  for  either  the  sal f "evaluat Ion  or  feedback  group  at  the  first 
checkpoint.     For  the  control  group  the  final  product  was  their  first  opportunity 
to  evaluate  their  own  work.    One  possible  eKplanatiQn  for  this  phenomenon  is 
that  as  students  go  through  a  coursep  they  acquire  an  understanding  of 
instructori-  eKpectations  and  how  well  their  perforrnances  meet  these 
expactat ions.     Students  then  adjust  their  performances  accordingly. 

It  was  anticipated  that  sel  f --eva  1  uat  ion  itself  without  any  instructor 
feedback  on  the  accuracy  of  the  evaluation  would  raise  the  correlation 
between  student  and  Instructor  scores.    The  analysis  revealed  that  self-^ 
evaluation  itself  had  no  beneficial  effects^  and  in  fact,  seemed  to  have 
a  negative  effect,    Self^evaluation  seemed  to  confuse  students  and  hamper 
their  ability  to  use  prespeclfled  criteria  to  assess  the  quality  of  their 
materials.    We  have  no  explanation  for  the  occurrence  of  this  effect  In. 
either  the  pilot  study  or  this  study.    There  Is  no  literature  on  which  to 
base  an  explanation  of  this  kind  of  result. 

The  effects  of  self-^evaluat Ion*  and  self=evaluation  and  feedback  on 
student  performance  is  difficult  to  gauge  In  this  study*    Students  In  all 
three  groups  achieved  a  grade  of  A  on  the  final  project  scoring  an  average 
of  55  out  of  a  possible  60  points.    There  was  little  variability  In  student 
performance , 

When  tied  to  student  confidence  in  their  materials,  there  was  a 
signfficant  difference  between  group  means.     Feedback  on  self-evaluat Ion  of 
earlier  material  seems  to  be  highly  effective  In  pronioting  improved 
student  scores  on  later  performances  when  student  confidence  in  materials 
was  incorrectly  high.    When  students  received  instructor  feedback  on  the 
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quality  of  their  materials  and  on  their  own  eyaluations,  they  had  the 

opportunity  to  learn  not  onlY  what  Instructors  thought  of  their  materials, 
but  how  their  self-evaluations  compared  to  instructor  evaluations  on  an 
objective  by  objective  basis.    This  technique  pushed  the  mean  scores  of 
students  In  the  unwarrinted  high  confidence  subgroup  3*5  points  above  the 
feedback  group  mean  from  5^.75  to  58.25*    This  tends  to  support  the  conclu- 
sions of  Kulhavy,  Yekovlch  S  Dyer  (1976)  that  feedback  Is  most  effective  when 
a  high  degree  of  student  confidence  in  materials  is  unwarranted.     In  the  self" 
evaluation  group,  the  difference  between  the  mean  scores  of  students  who  had 
incorrectly  assumed  their  materials  were  good  and  the  entire  sel f^eval uat Ion 
group  was  minimal,  lending  more  support  to  the  conclusion  that  the  addition  of 
feedback  on  the  self-evaluations  is  beneficial  when  learner  confidence  In  their 
work  is  incorrectly  high. 

When  student  confidence  in  materials  was  correctly  low,  feedback  on  self^ 
evaluation  seemed  to  have  a  negative  effect  on  performance,  lowering  the 
subgroup  means  from  5^^7S  to  ^9*50.    These  results  contradict  the  hypothesis 
stated  earlier  that  feedback  on  self-devaluations  is  effective  for  all 
students j  but  support  the  findlngi  of  Kulhavyj  et  al.  who  concluded  that  when 
student  confidence  was  correctly  low,  they  were  confused,  and  feedback  was  not 
effective*    The  results  of  this  study  Indicate  that  when  students  were  unsure 
of  their  work,  additional  feedback  only  caused  more  confusion. 

Duel   (1968)  and  Irwin  (1973)  both  concluded  that  sel f =evaluat ion  did 
improve  student  performance.    Due  to  the  celling  effect  noticed  In  this 
study  J  the  current  results  can  neither  support  their  conclusions ^  nor  refute 
the  findings  of  McEowen,  who  found  no  Improvement,  and  Fried  (1965),, who  found 
Improvement  in  some  situations.     It  Is  clear  that  the  effects  of  self-- 


evaluation  on  student  performance  would  possibly  be  more  informative  under 
conditions  In  which  student  grades  are  normally  distributed* 

Although  the  research  seems  to  support  the  results  of  the  Kulhavy,  et 
al*,   (1976)  research  cencernlng  the  effects  of  feedback^  continued  research 
in  this  area  is  also  indicated  in  order  to  clarify  the  effects  of  feedback 
In  learning  situations  in  which  student  performances  are  complex  and  student 
confidence  is  difficult  to  gauge. 

There  were  many  questions  raised  during  this  study  that  will  need  to  be 
investigated  In  future  studies.    Two  were  of  particular  Interest*    The  first 
question    relates  to  the  unexplained  negative  effects  of  self-eva 1 uatlon  only 
on  student  evaluations  of  their  own  work*    The  researchers  expected  this  acti-^ 
vity  to  increase  the  ability  of  students  to  assess  their  own  work.    When  in^ 
structors  appeared  to  ignore  student  evaluations,  students  were  not  helped* 

A  second  question  of  interest  is  where  to  begin  instruction  for  students 
who  are  unable  to  use  prespecified  criteria  to  improve  the  quality  of  their 
work.     If  students  cannot  perform  specified  tasksp  cannot  use  ©bjectivei  and 
criteria  statements  to  evaluate  their  own  workp  and  cannot  use  instructor 
feedback  to  improve  their  productSi  then  the  problem  may  be  that  these  students 
do  not  possess  prerequisite  skills  for  the  Instruction*    A  feedback  strategy 
that  includes  Information  on  current  initructton  and  on  prerequisite  skills 
Instruction  should  be  tnveitigated  In  an  attempt  to  Identify  the  best  strategy 
for  this  particular  subgroup  of  students* 

If  students  can  learn  to  use  preipecified  objectives  and  criteria  effect^ 
Ively  to  evaluate  and  shape  their  performances ,  the  case  for  the  use  of  In- 
structional objectives  Is  made  even  stronger,    Thent  not  only  will  students 
know  precisely  what  Is  expected,  but  they  will  be  able  to  tell   if  and  when  they  are 
able  to  perform  the  specified  tasks. 
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If  student  self-evaluation  can  be  developed  to  an  acceptable  levgl,  then 
time  spent  on  Instructor  feedback  can  be  greatly  reducedj  freeing  Instructors 
for  other  Instructional  tasks  during  the  educational  process. 
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Planning  Oocyment  Evaluation 


NAhI 


Oirectidnf       Uiing  a  I  to  10  icale,  v/ith  1  being  ths  lowiit  and  10  che 
highest^  evaluate  each  lacttsn  of  the  planning  document. 
PleaSa  use  only  whole  numbers  (no  Fracctsns),     iaie  your 
evaluations  on  the  crfcsria  suppl itd  for  aach  section. 
Thank  you.     Each  arei  has  a  blank  to  account  for  any  other 
cons !^ari£ ions  wniGh  may  eome  up. 


^'"'^^  ^^^^^    '  ^  Cantfint  Outline 

a)  Is  the  CQntent  of  your  outline  fully  ejcplained? 

b)  is  the  skill  or  cQncepc  bflng  cagghE  covered  fn  its 
en  t [ re  ty? 

-) 

2  -  Heed  -  Purpoii 

a)  Is  the  statafnent  of  the  pufpose  a  clear  itatamenc  of 
the  teacher*!  goal? 

b)  la  the  need  for  the  unit  clearly  deScrfbed? 

^'^'^^  ^.--^..-^    3  "  Audience  Analysts 

a)  Does  the  audience  analysis  contain  some  gr  all  of  the 
fo I lowi ng  : 

1)  class  size? 

2)  descrfptign  of  ability  level  of  ftudanEs? 

3)  deicriptign  of  racial  makeup  of  the  class? 

^)  deSGripcign  of  diFferenE  io^io^iconofTilc  groups 
found  In  claii? 

5) 

^^^-0^  .^.^^   ^  "  Objectives 

b)  Are  object ives  meaningful? 

b)  Are  objectives  related  to  unit  of  initrucEion? 

c)  Are  object  I  vas  % 

1)  icudent  oriented? 

2)  behavjoral!y  stated? 

d)  Do  objectives: 

Ij  specify  conditions  of  per formance? 
2)  ipicify  standards  of  perfoi'mjnce? 

e) 

^^"'^^  --...^   5  ^  Media  Sa Action 

a)  Does  media  have  potential   for  helping  learners? 

b)  Are  there  other  more  appropriate  m£.;ia  which  could  mike 
the  instruction  more  effictive  or  trficlent? 

c)  Are  there  any  special  c  i  rcurtistances  £hat  affect  media 
selection  which  have  not  been  considered? 


(MlQ)  _  6  »  Ut| 


I  ia  ti  on 

«)     Is  the  process  for  implenient  ing  the  unit  clear  and  senilbl 

b)  Are  all   the  materlali  utn  Ized? 

c)  Is  the  teaching  strategy  coniiitant  with  the  audience, 
©bjectiveSj  purpose,  and  t ime  tons Er al n^s? 

d) 


ss»i-7a  I "  I 

Maweriils  Evaluation 

NAMi 


Difections        Using  a  1   to  IQ  SsaU,  with  I  being  the  lowest  and  10  thg 

highest,  evaluate  ^ach  aspect  of  the  rnitefials  produced  for 
the  media  ppgduct,    Please  uis  only  whole  number^  (no 
ffaGCions).    iase  Your  evatyition  on  the  CFiteria  supplied 
for  eac^  aspect.    Siank.s  are  left  to  ^ccouht  for  any  other 
cons  idgracions  vmjcri  itiay  come  up,     inanK  you. 

TECHNICAL  qUALlTHS  CRlTEFJA 

(1-10)    1  .  Ltttertng 

a)  li  leUerIng  Isgible? 

b)  Are  lecters  neitly  done? 

c)  Are   letters  ind  words  appraprf ^tety  i?a^€d^ 

d)  Are  there  anV  unwanted  spots  of  Ink  or  [nk  smears? 

e)  .....  ,   .  ...... 


f) 


Are  there  anV  unwanted  ipocs  or  ihk  or  \ 
Have  guideMnes  been  complately  erased? 


(]-!0)    1.  iilyscratton 

a)  Is  Inking  neat  and  accurata? 

b)  is  thers  sufflcienC  contrast  between  or ! g » 1 /boa rd? 

c)  Have  ill  pencil/stray  marki  bein  cleaned  off? 

d)  Does  enlarggment  fit  board  properly? 
e) 

(I-IQ)  ^   3*  Mounting 

a)  Is  visual  completely  adhered  to  backing? 

b)  Is  visual  trimmed  neat  and  clean? 

c)  Have  registracion  Tiaras  been  erased? 

d)  Has  e5<ceii  HT-Si  Sealamlni  and  rubber  csmenc  been 


re 

a)     Is  visual      11 -pos  1 1  f  oned? 
f) 

(1-10)    Transparency  ^^aklng 

a)  Are  lines  soUd  when  projected? 

b)  Are  colored  areas  soUd  and  clean? 

c)  Is  hinging  secure  and  workable? 

d)  Is  transparency  neat  and  dean? 
a) 

(;-lQ)    5'  Conicruccion 

a)  Is  product  cOnSfructsd  soundly  or  hdld  together 
securely? 

b)  Hiii  adequate  caution  been  taken  to  aiiur^  product  will 
stand  up  to  clafsroon  use? 

c) 

(1-IQ)  ^   6.  Appearance 

a)  li  the  prcdyct  neat? 

b)  Have  any  errors  been  qade  which  detract  fron  the 
product's  Overall  appearance? 

c) 

^'^^^0)  __    7.    Other  (pUaie  liit) 
a) 
b) 


SS-1-78 

Hass  i^fedia  Project  Evaluaclon 


NUM 


NAME 


Dlrecdoni       Using  a  1  tg  10  scalt*  with  1  being  th^  \m@%t  and  10  thm  highest* 
evaluics  each  ispecE  of  che  mats  medja  product.    Pleas©  use  only 
whole  numbers  (no  fractsans).    iase  yQur  tvaluation  on  the  cri terra 
supplied  for  each  ispae".    SUnki  are  left  to  accoun  C  For  any  Qther 

cons  i  derd  L  i  una   Jtlilch  hmy  tatfie  up  .     Tiidrik  yuu, 
HASS  MEDIA  PROJECT  CRITERIA 


(1"IQ)  1.  Netd-PyrpQii 

i)     Is  che  scatenient  Qf  Che  Durpose  a  clear  statsnent  &F 

teacher's  goal? 
b)     Is  Che  -efid  far  zhm  yn*-  clearly  described? 
c) 

^'"^^^^  ..^^    2.    Audience  Analyifs 

i)    Does  thm  ludiance  anal /sis  concafn  some  or  ill  of  the  foHowrng 
1)    class  size? 

1)    description  of  ability  level  of  itudenci? 

3)    description  of  racial  fnakeup  oF  the  class? 

h)    description  of  different  socig^economic  grgups  found 


in  class? 


5) 


^''^^^  ___  ____    3'    Oeicripcion  of  Prografn 

a)  Title 

b)  Tine 

c)  Where  can  It  be  vfeweal 

d)  Length 

e)  Hedium  (TV,  Fi Im,  ^tc J 

n 

('^^^^  ^-     Synopsis  of  Program 

a)  Is  program  clearly  described? 

b)  Is  focus  of  lesion  id^ntifiadf 

c)  . 

(^^^O)  _____   S.     Inscructional  Activities 

a)  Do  preprogram  actrvitiel         up  -3  Framework  fgr  viewing? 

b)  Is  the  relationship  between  priprogra^  iccivlciei,  pos  tproQr^ifTi 
r'sctivitjes  and  the  progrj.ii  iSielf  cUir? 

c)  Do  activitits  h^lp  learner  aCtafn  the  objective  of  the  lesson? 
^) 

^^'^0)  -  6'  Objecciyes 

a)  Are  objectives  meanfngful? 

b)  Are  objectives  rslatid  to  unic  of  instruction? 

c)  Are  objecti vei : 

1)     student  grient^dl 
2}    behaviorally  stated? 

d)  Do  Objectives^ 

1)  specify  condicians  of  performance? 

2)  speci f y.  standai^ds  performance? 
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An  Empirical  Analysis  of  the  Characteristics  of  Performanct  Objectives* 


Vernon  Gerlach 
Arizona  State  University 

Richard  F.  Schmid 
Concordia  University 

Purpose 

The  concept  of  instructional  systena  design  is  inextricably  intertwined 
with  the  concept  of  performance  objectives.    The  latter  are  referred  to  by 
many  different  terms,  such  as  behavioral  objectives,  teTMnal  leartiing 

every  model  for  the  design  of  instructional  systeM,  the  performance 
objective  Is  the  point  of  departure;  decisions  regardljig  neaaurementj 
evalimtlon,  production ^  and  sequencing  (to  mention  only  the  obvious)  are 
''ob  j  active-based ,  " 

Most  advocates  of  performance  objectives  agree  that  an  adequate  objective 
consists  of  three  basic  cpmponents.    Consider  this  performance  dbjeetivei 
To  measure  the  diameter  of  a  solid  cylinder ^  given  a  tubular  ffilcrometer,  to 
the  nearest  .001.'"    Note  Che  three  cpmponentsi     (1)  the  verbs  which  states 
what  the  trainee  will  be  able  to  do  after  his  training  (to  measure) ;  (2)  the 
condition  of  performance ,  ^Ich  states  the  circumstances  under  which  the 
action  occurs  (given  solid  cylinders  to  be  TOeasured  and  a  tubular  micrometer) i 
and  (3)  the  criterion,  which  states  the  stmdard  of  performance  (to  the 
nearest  ,001") • 

At         three  reaent  AEGT  conventions ^  we  have  reported  the  results 

of  our  research  in  the  area  of  the  behavioral  characteristics  of  behavioral 
objectives*    Basically ^  we  have  been  stteinptlng  to  find  answers  to  the  question, 
to  what  degree  do  the  several  components  Influence  readers^  perceptions  of  the 
observability  of  various  performance  objectives  or  of  the  components  of  such 
objectives?    The  findinga  reported  have  met  generally  accepted  criteria  for 
reliability.    However^  our  data  provided  no  basis  for  answering  the  question, 
to  what  degree  do  these  findings  apply  to  the  area  of  military  instructional 
systems  design,  as  opposed  to  teaching  and  Instructional  development  in 
public  schools?    A  simple  correlational  study  was  designed  to  provide 
answers  to  this  question* 

Rationale 


Dano  and  Jenkins  (1969)  found  that  many  verbs  which  elementary  and 
secondary  school  curriculum  experts  and  instructional  d.eslgners  recommend 
for  instructional  planning  were  not  perceived  by  educators  to  be  "observable,*' 
They  concluded  that  verbs  used  In  behavioral  objectives  are  selected  for 
conilstency  of  usage  rather  than  for  tnaKlmum  observability •     Gerlach  (1974) 


^Thls  paper  is  based  in  part  on  research  fiffLded  by  U,S,  Air  Force  Office 
of  Scientific  Research  Gr?nt  tfTS-'ZgOOs  awarded  to  Arizona  Stace  University. 
Vernon  S*  Gerlach  is  the  principal  Investigator* 
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replicated  this  study.    He  found  a  correlation  of  *90  between  the  1969 
results  and  his  own. 

Gerlach  and  Barron  (1974)  conducted  a  study  to  determine  whether  or 
not  statements  of  conditions  or  statements  of  criteria  ciight  alter 
individuals'  perceptions  of  the  observability  of  verbs  used  In  co^lete 
behavioral  objective.    For  exaraplCj  would  a  verb  which  received  a  low 
observability  rating  when  rMked  in  Isolation  receive  a  higher  rating  if 
presented  within  a  complete  objective  with  a  precise  etatement  of  condition 
and/or  atatement  of  criterion*    Their  results  demonstrated  that  the 
condition  and  criterion  Influance  not  only  the  ratings  of  verbs,  but  also 
they  influence  the  ratings  of  complete  objectives  oiore  than  the  verb  does. 
This  finding  was  strengthened  by  a  subsequent  study  (Barron  Bikd  Gerlaah,  1975)5 
which  also  revealed  that  the  use  of  two  scales,  "most  observable  to  least 
observable"  and  "preciss  to  vague,"  provided  more  reliable  and  memlngful 
data  than  did  the  use  of  two  other  scales ,  "specific  to  general"  and  "concrete 
to  abstract*" 

Haygoodi  (ferlach,  and  Wigand  (1976)  extended  this  line  of  investigation 
in  a  study  designed  to  determne  whether  or  not  the  direct  object  In  an 
objective  slpiificantly  affected  the  rating*    They  found  that  there  was  no 
difference  between  subjects*  perceptions  of  objectives  which  contained  actual 
direct  objects  and  gtatements  which  contained  symbolic  representations  of 
direct  objects  (i.e.,  '*x"  and  "y"  were  substituted  for  actual  direct  objects)* 
However  J  they  did  find  that  varying  the  actual  direct  objects  did  produce 
sipiif leant  changes  in  raters*  perceptions  of  the  observability  of  the 
objectives. 

These  studies  have  led  us  to  conclude  that  emphasis  on  any  one  component 
of  a  performance  objective  is  not  warranted*    This  finding  is  inconsistent 
with  the  injunctions  and  e^chortations  foimd  in  mostp  if  not  all^  manuals  and 
guidelines  for  instructional  systems  design.    The  studies  clearly  indicate 
that  empirical  evidence  is  needed  to  detemine  how  an  objective  should  be 
constructed  in  order  to  mascinlge  the  possibility  that  the  reader  will 
perceive  it  as  describing  an  observable  event. 

However,  all  the  studies  cited  above  used  educators  or  students  in 
colleges  of  education  as  iubjects*    Generalising  the  findings  to  other 
populations  would  be  warranted  only  if  a  high  relationship  could  be  foimd 
between  the  results  of  the  studies  cited  and  the  results  of  a  replication 
using  another  populstion*    Specif ically,  it  was  hypothesized  that  the 
findings  could  be  generalised  to  two  types  of  military  personnel  i  trainees 
and  trainers  (including  instructional  systems  designers) , 

Procedure 

The  first  replication  used  30  imdergraduate  pilot  trainees  (UPTs)  at 
Williams  AFB,  Arizona.    They  were  administered  an  instrument  containing  123 
verbs  which  each  respondent  was  asked  to  rate  on  a  qulntile  scale  from  most 
observable  to  least  observable.    This  Instrmnent  was  eKactly  the  same  as 


that  used  by  Deno  and  Jenkins  (1969)  and  Garlach  (1974),  with  one  additions 
the  previous  insttuBient  contained  99  verbs  foimd  In  school  currlculai  the 
present  instruffent  contained  all  these  pLui  23  verbs  found  in  USAF  Under-- 
graduate  Pilot  Training  syllabi*    A  second  replication ,  using  the  same 
instruments  WS  conducted  at  Luke  APB^  ^^ere  17  members  of  four  instruc- 
tional systems  devalppment  teams  served  as  subjects. 

Results 

The  correlation    between  the  ratings  of  the  UPTs  aad  the  results 
reported  by  Deno  ^nd  Jenkins  was  ,90.    The  correlation    betwean  the  ratings 
of  the  ISD  team  members  and  the  Deno-=Jenkins  rasults  was  ,91^    The  correlation 
between  the  Wt  and  ISD  ratings  was  ^89*    The  mean  rating  of  the  99  verbs 
used  by  Deno  and  Jenkins  was  3*06,    UPTs  gave  these  same  verbs  a  mean  rating 
of  2,57,  while  ISD  team  members  rated  them  2.51.    The  23  verbs  ^yhlch  were 
randomly  selected  from  ATC  training  materials  were  given  mean  ratings  of 
2 •AS  and  2*50  by  the  UPTs  and  ISD  team  members ^  respectively* 

Conclusions 

The  correlations  obtained  are  sufficiently  high  to  warrant  our  concluding 
thats  with  respect  to  the  content  of  this  study ^  we  are ^dealing  with  highly 
similar  if  not  identical  populations.    Consequently ^  we' are  convinced  that 
the  findings  of  earlier  studies  concerning  the  three  componetits  of  perform^ce 
objectives j  as  well  as  the  findings  concerning  direct  objects^  are  general-- 
lEable  to  the  conteKt  of  Air  Force  training. 

It  appears  that  some  of  the  operational  diffictiltles  carrently  encoimtared 
by  field  ISD  teams  (espeeially  for  co^leK  perceptual  motor  skill  training) 
would  be  reduced  if  existing  manuals  were  revised  and  new  ffiatiuals  written  to 
incorporate  the  findings  concerning  the  observability  of  verbs.  Furthermore, 
the  three  factor  model  of  objectives  (verb,  conditions ^  and  criteria)  should 
be  eKpanded  to  include  the  direct  object. 

Tables  showlni  the  means  and  variances  of  the  ratings  for  each  of  the  99 
verbs  for  each  of  the  three  subject  populations  will  be  distributed  at  the 
session*    Likewise ,  the  tables  showing  the  obtained  simple  and  multiple 
correlations  will  be  made  available , 
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Abstract 

The  Politics  of  Instructional  Innovation  in  Higher  Education:  A  Cross 
Cultural  Analysis. 

Higher  education  administrators  have  supported  improvements  in  teaching  by 
creating  special  funds  to  support  selected  projects.    These  projects  have 
not  been  universally  successful:  sonio  projects  have  folded  after  the 
initial  funding  ceased,  v/hlle  others  never  really  started.    This  study 
vas  an  attempt  to  test  the  strength  of  path  models  for  successful 
instructional  development  projects  in  two  different  cultures  and  academic 
structures.    In  a  series  of  Australian  projects,  administrators,  instructional 
developers  and  project  directors  were  asked  to  coniplete  the  same  instrument 
that  had  been  used  in  the  United  States  of  America.    The  most  Important 
findings  have  been  the  completely  different  factor  structures.    Where  the 
United  States  respondents  saw  the  Issues  as  provision  of  support  services , 
organisational  support  vnth  good  administrative  systerns.  faculty  Incentives 
(motivation) j  and  financial  resources ^    the  Australian  respondents  saw  the 
major  issues  as  provision  of  support  and  administrative  services^  status  of 
the  project  internally  and  externany  (including  some  similarity  to  faculty 
incentives  above),  positive  and  clear  innovative  climate  and  expertise  and 
skills  of  the  member  of  faculty  to  carry  out  the  project. 


"Most  of  the  innovation  that  occurs  could  best  be  described  as 
fiddling.    Cummulative  assessment  is  brought  in  here,  open  book 
examining  there i  a  game  or  two  somewhere  else,  TV,  CAI  or  carrels. 
These  are  healthy  signs  of  dissatisfaction  and  uncertainty,  but 
they  tend  to  relieve  the  initiators  of  the  need  to  face  the 
fundamental  re-appraisal  of  objectives,  course  design^  assessment 
and  outcome  that  experience  suggests  is  both  required  and  possible", 
(Wilson,  1973). 

Innovation  and  change  are  fundamental  to  the  process  of  instructional 
development,  and  in  recent  years  administrators  1n  higher  education  have 
supported  curriculum  iiTiprovements  by  funding  selected  projects.    However ^ 
these  projects  have  not  been  universally  successful:  some  projects  have 
folded  after  the  inital  funding  ceased;  some  when  the  project  director 
left  for  a  new  position;  and  some  ran  into  organizational  and  administrative 
problems  that  prevented  them  from  ever  starting. 

In  a  review  of  research  on  curriculum  and  instruction/  Fullan  and  Pomfret 
{1977}  established  that  the  major  emphases  have  been  on  the  adoption 
decision  before  a  change  is  implemented  and  monitoring  the  results  of 
change*    Recently^  more  research  has  focussed  on  the  process  of  implementation* 
Loucks  (1978)  and  Hall  (1977)  have  described  their  efforts  to  categorize  the 
level  of  use  of  an  innovation  and  thereby  predicting  future  stages  of 
development.    Lawrason  and  Hedberg  (1978)  have  presented  a  path  model 
to  predict  instructional  development  success  on  the  basis  of  institutional 
committment  and  personal  expertise  of  the  academic  staff.    This  change  In 
focus  provides  the  instructional  development  specialist  with  an  ordered 
array  of  factors  that  can  be  used  to  successfully  manage  instructional 
innovations. 

In  commenting  upon  his  experience  at  tv/o  universities,  Gropper  (1977) 
advises  the  Instructional  designer  to  be  sensitive  to  the  bias,  fears 
and  misunderstanding  on  the  part  of  the  academic  staff  and  administration 
unfamiliar   with  instructional  design  and  change.    The  current  modjs  of 
teaching  and  decision-making  are  firmly  entrenched  within  th  organisational 
structure.    Innovation  if  it  is  to  occur,  must  be  achieved  subtlely 
and  gradually.    Support  for  the  project  must  therefore  come  from  those 
helping  to  achieve  the  change.    If  the  project  is  supported  by  the  central 
instructional  design,  production  and  evaluation  staff,  then  this  will  help 


the  initial  development  of  the  project  considerably,  even  If  funds  are  wt 
available  to  support  major  expenses.    !n  fact,  support  staff,  resources  and 
rewards  can  be  used  to  ensure  the  continuation  of  good  projects.  Gropper 
observed  that  a  central  facility  with  the  specific  expertise  in  instructional 
design, evaluation  and  production  should  bQ  available  centrally  for  the 
support  of  academic  staff,  and  that  the  administration,  academic  boards  and 
prornotion  committers  should  reward  the  effective  teaching  and  curriculum 
design  of  academic  staff  participating  in  instructional  design  projects. 
While  some  administrations  rewarded  curriculuiTi  re-design  efforts,  by 
far  the  majority  of  projects  surveyed  in  one  study  (Lawrason ,1977)  did  not 
have  any  central  Instructional  developiiicnt  support. 

Without  support,  what  then  acts  as  incentives  for  instructional  development? 
Spitzer  (1977)  asked  academic  staff  in  sixty  institutions  throughout  the 
United  States  why  they  undertook  instructional  development  projects.  Spitzer 
found  the  indirect  organizational  incentives  (administration  conimittment, 
competent  Instructional  development  staff ,  etc.,)  v/ere  the  most  clearly 
developed  and  recognizable  success  factors.    Direct  incentives  (payments, 
release  time,  etc.,),  recognition  (letters  of  appreciation,  titles,  etc.,) 
and  personal  satisfaction  (achievement,  striving  for  excellence,  etc.,)  were 
of  lesser  Importance  and  demonstrated  less  consistency  in  responses  appearing 
to  be  specific  to  the  Individual  or  the  motivation.    The  evidence  of  success 
of  most  projects  as  Spitzer  pointed  out  was  based  upon  intuitive  rather  than 
research  evidence. 

Lawrason  and  Hedberg  (1977)  identified  the  interrelations  among  kc;,' 
instruction  design  factors.    From  the  instructional  design  literature  they 
assembled  a  series  of  factors  that  were  considered  to  contribute  to  project 
success.    These  factors,  institutional  commitment,  faculty  rewards, 
instructional  design  staff  expertise,  campus  AV  production  services,  program 
evaluation  and  faculty  interest  in  innovation,  were  ranked  by  academic 
staff,  administrators  and  instructional  designers.    While  administrative 
commitment  to  instructional  development,  primarily  through  financial  support, 
vyas  extremely  important,  respondents  tended  to  group  factors  according  to 
highly  individualized  concerns,  and  the  factors  perceived  as  contributing  to 
success  were  related  to  the  respondent's  world  view,  and  not  to  the 
respondent's  ecadcmic  or  administrative  rank.    The  groups  of  rcspondGnts 
so  formed  were  characterized  by  their  views  on:  administrative  commitment, 
instructional  development  policies  on  curriculuni  change,  and  personal 
satisfaction  from  undertaking  the  project.    In  this  study  the  respondents 


wei-e  asked  to  reply  'in  general  and  no  attempt  was  rnadL'  to  relate 
iristructional  developniDnt  succgss  to  specific  projects. 

In  a  later  study,  (Lawrason  and  Hedberg,  1978)  respondents  were  asked  to 
examine  a  particular  instructional  design  project  and  coiniiient  on  its 
success  or  failure  and  the  factors  that  contributed  to  this  outcome,  The 
earlier  factors  were  refined  and  further  sub-divided  to  produce  an 
instrument  that  examined  four  major  areas  1n  fifty  items;  the  provision 
of  instructional  design  and  production  support  services,  enthusiasisni 
of  thp  anademic  staff  for  chanoG.  provisiOfi  uf  resources  such  as  money  and 
equipment^  the  organisational  climate  that  would  hinder  or  facilitate 
innovations  and  changesp  and  items  related  to  student  interest  and 
behaviour.    One  third  of  the  respondents  Indicated  that  their  project 
was  initiated  by  a  member  of  the  conege  administration  and  this  fact 
alone  was  Instrumental  in  achieving  a  successful  outcome.    Under  these 
conditions  administrative  commitment  was  active  and  vigorous,  Unfortunately 
no  clear  distinction  v/as  made  between  dlfGCtive  and  participative 
generation  of  the  project,  thus  the  role  played  by  the  administrator  is  not 
alv/ays  clear.    Four  measures  of  project  success  were  collected:  an  overall 
estimation  of  project  success,  the  likelihood  of  the  project  continuing 
if  funding  ceases ^  the  lik&l Jhood  of  the  project  continuing  if  the  project 
director  leaves  the  institution  and  the  effect  of  the  project  on  student 
enrollments,    Lav/rason  and  Hedberg  (1978)  found  that  the  lik^ilihood    of  the 
project  continuing  if  funding  ceased  or  the  director  left,  w4re  predicted 
by  the  Institutional  and  inst  fictional  desion  factors  surrounding  the 
project.    The  other  tv/o  dependent  measures  were  not  predi  "ted  by  the 
same  factors.    The  overall  estimation  of  project  succp^    was  never  negative^ 
whereas  the  two  1 Ikel 1hood-of-cont1nuation  measures  appeared  to  be  less 
positive  and  had  a  greater  variance.    This  may  be  due  to  an  inherent 
limitotion  of  this  type  of  study,  the  respondent  was  often  the  project 
director  and  initiator  which  might  indicate  a  degree  of  ego-involvement 
with  the  outcome. 

The  path  models  determined  for  the  Lavvrason  and  Hedberg  study  (figure  1) 
Included  both  independent  variables  based  on  the  nature  of  the 
project/repondent,  and  composite  intervening  variables  derived  from  a 
cluster  analysis  of  the  fifty  item  questionnaire. 
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The  path  models  were  based  upon  a  sample  twenty-six  instructional 
developmont  projects  undertaken  in  six  universities  and  colleges  throug 
the  Delaware  Valley,  Pennsylvania. 


Purpose  of  the  curtopiJtydl 

This  study  sought  to  apply  the  path  models  of  figure  1  to  a  similar 
instructional  development  environmant  using  a  sample  drawn  from 
Australian  Higher  Education  InstitutiQns.    As  with  the  Lawrason  and  Hedberg 
(1978)  study,  this  comparative  study  is  based  upon  the  experience  of 
academic  staff  who  have  tried  to  redesign  a  course  and  the  factors  they 
felt  were  important  to  the  success  or  failure  of  their  instructional  project, 

Some  parallel s  might  be  drawn  between  the  two  sets  of  samples. 

1.  Both  samples  were  predominately  undertaking  proaects  to 
develop  their  own  course  of  approximately  seinester  length. 

2.  Host  respondents  categorized  themselves  as  Departmental 
Chairman  or  Tenured  Academic  Staff.    (It  is  possible  that 
the  lower  frequency  of  Untenured  Staff  responses  is  due  to 
their  chanying  positions), 

3.  The  most  common  roles  played  by  respondents  were  as  project 
director  and  instructional  designer. 


MetlTOd 


The  r^espondents  were  administrators,  project  directors^  academic  staff  and 
educational  support  staff  at  two  higher  education  institutions  in  Victo^^ia, 
Austral ia^  and  six  higher  education  institutions  in  the  Northeasterri  United 
States.    After  each  project  had  been  identified  through  educational  research 
officers  and  through  previous  studies,  the  project  director  was  called  on  tfie 
telephone  and  asked  if  he  or  she  and  co-v/Qfkers  v/ould  reply  to  the  questionna ir^s* 
Thfi' questionnaire  was  structured  into  three  sections:  Part  1  identified  the 
general  nature  of  the  project  and  the  person  respondlngs 

1  The  title  and  objective  of  the  project; 

2  The  size  of  the  project  (PSIZE); 

3  The  acadeniic  rank  of  the  rispondent  (RANK); 

4  The  part  played  by  the  irespondent  in  the  project  teairi  (PA?j); 

5  The  rank  of  the  person  who  initiated  the  project  (prfUT). 

Part  II  of  the  survey  consisted  of  a  BO-ltem  questionnaire  (Appendix  A)  which 
n^asuredthe  perceived  Importance  of  a  wide  range  of  factors  contributing 
to  the  outconne  of  an  Instructional  development  project.    Respondents  were 
asked  to  rate  each  of  the  50  items  on  a  scale  of  l(low  priority)  to  7 
(high  priority).  The  initial  path  models  had  been  established  with  the 
United  States  data  (Lawrason  and  Hedberg ,1978)  and  the  composite 
Intervening  variables  derived  from  the  50  itGms  were: 

1  The  Importance  of  the  relationship  between  the  support 
staff  and  the  project  members  (SUPPORT)* 

2  The  financial  incentives  or  intrinsic  motivators 
available  to  faculty  project  members  (FACINCEN); 

3  The  financial  resources  provided  by  the  administration 
for  the  funding  of  the  project  (RESOURCE);  and 

4  The  importance  of  organizational  facilitation^  i*e.  the 
interpersonal  relationships  between  the  administration 
and  project  members  (ORGFAC), 

Part  III  of  the  survey  required  respondents  first  to  rate  the  overall 
success  of  their  project  on  the  basis  of  four  specific  criteria.  The 
final  two  responses  requested  subjects  to  verbally  describe  the  "major 
factor  which  facilitated  success"  and  the  '^niajor  factor  which  limited 
the  success"  of  the  project*    The  four  specific  criteria  by  which  respondents 
rated  the  success  of  their  projects  were  considered  as  the  dependent 
O  variables  in  the  study. 


The^  were:  ,, 

1  A  global  rating  of  the  overall  success  of  the  project  (SUCC); 

2  The  likelihood  of  project  continuation  after  funds 
ceased  (FUND)i 

3  The  Tikelihood  of  project  continuation  after  the  original 
director  left  (DIR);  and 

4  The  effect  of  the  project  upon  student  onrollments  (STUD). 

Scores  for  the  composlto  factor  variables  (SUPPORT,  FACINCEN,  RESOURCE,  and 
ORFAC)  were  used  1n  a  standardized  form  1n  a  regression  analysis  to  deterniine 
path  co-efficiants.    On  the  basis  of  the  Lawrason  and  Hedberg  (1978)  study 
the  hypothesized  relationships  are  illustrated  in  Figure  1.    The  Australian 
data  was  analysized  by  reference  to  these  previously  established  path  inodcls 
and  then  a  factor  analysis  of  the  intervening  variables  was  attempted  to 
confirm  the  factor  pattern  determined  froni  the  U.S.  data. 

Results 


The  previous  study  was  based  upon  38  respondents  representing  26  instructional 
development  projects  in  United  States  institutions.    In  this  Australian 
comparison  27  respondents  represent    22  projects.    While  some  difficulty 
was  experienced  in  identifying  projects,  an  Qi%  response  rate  was  achieved 
in  the  two  Australian  Institutions. 

The. nature  of  the  surveyed  instructional  projects  was  characterized  by  the 
size  of  the  projects  the  respondent's  academic  rank,  the  part  the  respondent 
played  and  the  rank  of  the  person  who  initiated  the  project.    (Table  1). 
No  differences  mre  noted  in  these  categDrles  with  the  eKceptlon  of  the 
person  who  initiated  the  project.    Significantly  more  academic  staff 
initiated  insLructiontil  projects  in  the  Australian  sample,  and  correspondingly 
more  administrators  (deans,  departmental  chairpfirsons)initiated  projects  in 
the  U.S.  sample.    This  may  be  due  to  differences  in  tenure  percentages  vn'th 
more  tenured  staff  In  the  Australian  Institutions,  or  more  fluid  departnicntal 
Administrative  structure  of  the  American  Colleges. 
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Comparisions  betv/een  the  two  groups  v/ere  made  on  the  50- item  questionnairs 
on  project  success  factors.    Using  simple  t  tests  for  differences  between 
meanss  11  items  had  i- values  exceeding  the  probability  of  M. 


The  respondents  1n  the  Australian  sample  cohslctered  the  following  aspects 
more  iiTiportant  to  project  success  than  their  U.S.  colleagues: 

11      Provision  of  administrative  rewards  (release  time^  money)  for 
.  academic  staff  engaged  in  instructional  projects. 

16  Academic  staff  felt  need  to  change  existing  course  design 
and  teaching  strategies, 

17  Commitnient  by  academic  staff  to  irnpl^:'  ,nt  and  evaluate 
.  the  project  as  designed. 

20      Good  personal  relations  between  the  rriiimbers  of  the  academic 
staff* 

22      Priority  given  by  support  services  to  full  scale  funded 
Institutional  projects* 

26  Efficiency  of  support  services. 

27  Positive  and  supportive  attitudes  of  support  service  personnel. 

28  Good  personal  relations  between  academic  staff  and  support  staff 
on  the  project, 

45  Necessity  to  Improve  Instruction  for  a  specific  disadvantaged 
student  population, 

46  Large  number  of  students  to  benefit  froni  the  project. 

Some  observations  should  be  made  about  these  results.    The  emphasis  on 
academic  staff  (Q's  1146,17,20)  is  likely  to  be  due  to  the  differences  in  the 
person  v/ho  initiated  th.  project.    More  academic  staff  were  responsible  for 
project  initiation  in  Australian  and  this  would  increase  the  importance  of 
responses  directed  at  acade^iiic  staff  bona vi ours.    The  Australian 
projects  fequired  greater  iiwHa  support  than  the  U.S.  projects.    This  would 
seem  to  be  appropriately  relected  in  items  ZZ.gGJJ.and  28.    One  might 
explain  the  high  rating  of  the  two  student  benefit  items  in  that  the 
importance  of  the  project  to  student  learning  either  in  terms  of  quality 
or  quantity  has  often  been  used  as  a  criteria  for  project  funding  in 
Australia,    Reading  the  project  descriptions  requested  of  each  respondent  in 
the  questionnaire*  it  appeared  that  Australian  projects  were  more  closely 
related  to  basic  instructional  problems,  while  the  U,S.  projects  appeared 
more  to      derived  from  the  respondents  research  interests.    However,  the 
item  relating  to  the  iniportance  of  the  project  in  generating  research 
O  findings  v/as  not  rated  differently  by  the  two  groups;  both  rated  it  low  (3,3) 
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compared  to  other  items.    Only  one  item  v/as  considered  more  important 
by  U.S.  respondents  than  Australian  respondents,  namely 

•36      Self-support  (budget)  of  project  after  initial  development. 

The  Australian  projects  v/ere  undertakings  that  most  often  h&d  institutional 
commitinent  to  carry  the  project  into  regular  funding.    This  was  not  always 
apparent  in  the  U.S.  projects. 

Fa c tor  c o n f  i  r ma  t i  o n 


The  Australian  responses  to  the  50-1tems  were  compared  to  the  factor 
structure  previously  determined  for  the  U.S.  data  (Lawrason  and  Hedberg,  1978). 
A  similar  factor  structure  was  not  supported.    The  only  siniilarity  in 
structures  was  the  continued  grouping  of  items  26,27  and  28,  which  describe 
the  efficiency  of  and  relationships  with  the  support  services.  When  attempting 
to  confirm  the  path  models  of  Figure  1   with  the  previously  established 
factors,  no  significant  paths  were  found  that  included  the  intervening 
variables:    FACINCEN,  ORGFAC,  SUPPORT,  RESOURCE. 

In  light  of  this  result  a  factor  analysis  ms  undartaken  to  investigate 
the  structure  underlying  the  Au;,tralian  rssponses  to  the  instrument.  This 
produced  a  solution  of  four  factors  v/hich  were  characteriaisd: 

1  The  importance  of  coinmunication  and  support  links  between 

project  director,  administration  and  support  staff  personnel  (ORGSUP)- 

2  The  status  of  the  projeri-  and  the  director  within  and  outside  the 
institution  ( STATUS )s 

3  The  climate  or  human  relationships  surrounding  the  academic  staff 
working  on  the  project,  and  their  ability  to  be  self  sufficient  (CLIMAT) 

4  The  instruction  design  expertise  of  the  project  dirGctor  and  the 
clarity  of  project  goals  (EXPERT) 

The  composition  of  these  intervening  variables  was  conflrmofi  by  a  cluster 
analysis  using  the  Veldman  H-group  program,  (Table  2).    The  simple  correlations 
between  the  variables  used  in  the  final  path  analysis  is  given  in  Table  3. 
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Overan  project  success. 


Responses  by  subjects  to  the  questions  about  overan  sucess  of  their  projects 
(SUCC,  FUND,  DIR  and  STUD)  did  not  differ  significantly  between  the  two 
groups. 
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From  the  dimction  of  the  differences  in  means  it  would  appear  that  US  projects 
are  more  dependent  on  the  director  for  continuation  of  the  project,  Hov/ever 
they  appear  to  generate  more  student  enrolrnents  than  their  Australian 
counterparts. 

In  v/riting  coniinents  about  their  projects  niost  Australian  directors 
eniphasized  the  grass  roots  nature  of  Instructional  Innovation*    While  many 
worked  as  part  of  teams 5  the  projects  v/ere  often  self-styled  with  loosely 
defined  structures.    By  far  the  most  important  factor  that  helped  the  project 
was  co^operatiDn  from  the  respondent's  colleagues^  this  v/as  iiiGntioned  ' 
by  two  thirds  of  the  respondents.    The  remaining  factors  were:  fundingp  positive 
student  reaction,  assistance  of  support  staffs  clear  objectives  and  time. 
More  serious  was  the  list  of  1  imitating  factors.    One  third  clainied  there 
v/as  a  lack  of  departmental  support  and  comniitment,  and  that  funds  v/ere 
insufficient.    Other  limitations  were  delays^  not  enough  people  or  not 
enough  time,    Tv/o  projects  v/ere  hindered  by  the  poor  leadership  and  lack  of 
expertise  of  the  director. 


Discussion 


Of  the  projects  1n  this  stiidy  over  half  (16)  of  the  respondents  were  v/orking 
on  single  course  or  snmll  projects.    This  contrasts  markedly  with  the  U,S. 
sample  where  the  majoi  ivy  of  respondents  (31)  were  working  on  projects  larger 
than  a  single  course.    This  pattern  would  seem  to  reflect  different  attitudes 
to  instructioncil  development  1n  the  two  countries.    The  Australian  respondents 
wore  primarily  working  on  their  own  projects^  while  U*S,  respondents  often 
worked  on  projects  initiated  by  another  person. 

Many  projects  generated  large  amounts  of  medlar-based  individual  study 
materials.    In  both  sanipleSs  the  academic  staff  often  undertook  the 
development  without  an  instructional  development  specialist  to  assist 
them.    Support  personnel  v^ere  usually  in  a  service  role^  providing 
graphics^  recording  a  tape,  or  consulting  on  problem  areas  only.  This 
eKclusion  of  trained  instructional  development  specialists  had  resulted 
in  some  wasteful  failures  and  where  specialists  were  available  the  lack 
of  good  client  relationships  appeared  to  alienate  the  Innovative  academic 
staff.    An  instructional  designer  could  help  these  projects  but  only    1f  he 
or  she  can  gain  acceptance  as  a  team  member  and  achieve  harmonious  relations 
with  the  academic  staff. 

It  is  interesting  to  note  that  only  one  respondent  in  the  two  groups  reported 
a  lack  of  success.    The  respondent^  a  chairman  of  a  department^  cited  the 
failure  of  the  administration  to  inform  the  project  director  of  the  degree, 
of  funding  for  the  project.    The  delay,  which  ran  to  three  years^  does  seem 
excessive- 

Lawrason  and  Hedberg  (1978)  found  that  of  the  dependent  measures  -  continuation 
of  funding  ceases  (FUND)  and  continuation  if  the  director  leaves  (DIR)  were 
more  important  predictors  of  the  successful  project.    The  overall  likelihood 
of  a  project  continuing  If  funds  ceased  was  ,76  for  the  U,S,  sample  and 
.78  for  the  Australian  sample.    On  the  other  hand,  the  1  livelihood  of  a  project 
continuing  if  the  director  left  was  .64  for  the  U,S.  sample  and  .76  for  the 
Australian  sample,  (The  likelihoods  ranging  from  0-unl1kely  to  1-certainty) , 
In  both  sample  these  tv^o  dependent  measures  were  important  In  the  path  models 
and  appeared  to  provide  better  estimates  of  the  'real'  outcome  of  a  project 
Although  the  difference  in  likelihoods  of  continuance  if  the  director  left 
was  not  statistically  significant,  there  is  an  indication  that  the  U,S. 
^  projects  are  more  heavily  ciependGnt  on  their  project  director  for  a  successful 


CQmpari^l^on  of  path  models 

As  previsouly  discussed  the  Australian  sample  did  not  conform  to  the  same 
factor  structure  nor  did  it  confirm  the  previously  determined  path  models 
(Figure  1),    The  only  overlap    between  the  factor  structures  is  the  consistent 
grouping  of  support  services.    The  remaining  factors  in  the  Australian 
sample  emphasize  human  relations  sknis^  expertise  or  competence  and  the 
standing  of  the  project  and  director*    The  factors  determined  on  the  basis 
of  the  U,S.  data  emphasiied  faculty  incentives,  project  resources  and 
administrativG  commitment  all  niore  external  concerns  than  the  more 
hesitant  ego-involved  items  that  were  grouped  by  the  Australian  sample* 

A  path  model  based  on  the  assumption  that  the  four  indenpent  variables 
(RANK,  PART,  PSI2E,  PINIT)  would  predict  the  four  intervening  variables 
(ORGSUP,  STATUS,  CLIMAT,  EXPERT)  which  in  turn  v^ould  predict  the  four 
dependent  variables  (SUCC,  FUND,  DIR  STUD)  produced  only  1 imted  paths 
(Figure  2),    Unlike  the  U*S*  samplep  the  overall  rating  of  project  success 
contributed  as  a  major  intervening  variable  in  the  study.    Some  slmnarities 
are    reassuring^  such  as  the  iinportance  of  the  academic  rank  of  the  respondent 
in  predicting  likelihood  of  project  continuation  if  the  director  left. 
Howeverj  the  direct  RANK  to  FUND  path  was  not  significant  in  the  Australian 
sample*    The  path  model  of  Figure  2,  does  concur  with  Spitzer  (1977)  when 
it  emphasises  that  the  climate  surrounding  the  project  is  an  important  link 
in  the  prediction  of  project  success. 

A  number  of  other  interesting  relationships  were  indicated  by  the  Australian 
data  (although  not  Included  1n  the  final  path  model)'    the  correlation 
between  organiEational  support  (ORGSUP)  and  the  STATUS  of  the  project;  and 
the  joint  prediction  of  instruction  development  expertise  (EXPERT)  by  the 
part  the  respondent  played  in  the  project  (PART)  and  the  size  of  the  project 
,  (PSI2E).    This  last  path  pointed  out  the  rather  damning  relationship  -  the  more 
important  the  part  the  respondent  plays  in  the  project  and  the  larger  its 
size,  the  less  important  Instructional  design  expertise  or  competence  is  rated. 
Maybo  it*s  simply  a  fact  that  large  projects  suceed  In  spite  of  the  lack 
of  expertise  of  the  innovative  academic  staff. 
Conclujion 

In  considering  the  most  essential  factor  influencing  the  success  or  failure 
of  projects,  it  is  interesting  to  note  that  rebpondents  place  prime  attention 
on  the  interest,  co-operation  and  skills  in  the  academic  staff  team  jnembcrs. 
O  It  is  evident  that  faculty  encjnqcd  in  dGVclopmont  projects  place  the  burden 
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of  successful  ImpleniGntation  upon  themselves  and  their  academic  colleagues. 
Over  half  of  those  rGsponding  in  both  saniples  to  the  question  'major 
factors  contributing  to  success- mention  the  initiative  or  skills  of  the 
project  director  or  other  academic  colleagues.    Far  fevver  list  administrative 
support,  support  services  or  even  funds. 

The  comparisions  made  in  this  study  should  raise  a  note  of  caution  to  any 
researcher  extrapolating  the  findings  of  one  successful  innovation  to  predict 
outcomes  in  a  different  educational  system.    Even  though ^  close  cultural 
parallels  exist  in  the  two  samples  chosen  for  this  studys  differences  have 
been  noted  in  the  degree  of  control  over  the  environment  of  the  project* 
Australian  respondents  were  more  concerned  with  academic  staff  behaviour 
while  U.S.  respondents  were  more  concerned  with  Institutional  reward  systems 
and  the  provision  of  resources.    The  results  of  this  study  might  be 
cansidered  a  first  approximation  to  the  many  factors. in  the  institutional 
environment  that  affect  instructional  development*    The  importance  of  good 
interpersonal  relations  at  all  levels^  mutually  defined  goals,  and  the  supportive 
rather  than  directive  role  of  the  development  specialist  avQ  essential. 
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Appendix  A 


Part  II 


fallowirig  luc:lDrs  cancQntf  lUitrj  lo  Iho  succoss  of  £in 
edunyiinrui!  dovaloprnent  projocl  w^^^"^         ^^  itution 
PUi^DO  rata  llio  i(npv>  tnn-o  of  uach  "  ■  .  ju  L^ilievo  it 

CDnlribuUJcJ     thff  miccoss  of  -r-v    ^-  \  cicmni  project  you 
huvn  idoniifiod.  C«rclD  thi>  m^mhdr  U\ui  most  closoly 
rcpre?ants  the  imponancu 
For  oKample; 

Rypicl  provisiori  of  typed  nianuscriptn 

1  Cluarly  duflfiu:!  aianni^ationjl  strLJClure  far  docision  making 
and  implUfneriUMion 

2  SniaU  numbnri,  of  nooplu  invjjlvtjj  m  tho  decision  making 
^ssocicjlod  wvicfi  the  praifcci 

3  Lnck  of  cntnpi'titivMnoss  ^ind  pQ^soi^nivennss  within  eKisting 
academic  dunortiriynts 

4  Personol  support  uf  top  ci^iminisU-jtorL. 

5  Provision  of  instruciinnal  projoct  'stnrt-up"  funds  trom  operatinfl 
budcjc-ts 

6  Continued  bml^oidry  suppart  uf  aJministrcjtors 

7  AdfniriiblfdliDn  concern  for  studont  onrolmunts 

B     Fij^n:ibility  of  adiniriicirntors  to  inluyrntfi  chango  Into  the 
ndfninistryt  ivG  systc^rn 

0     Positive  lu<idarship  skills  of  cidrniniMrntors 

10  Initiation  of  projects  hy  adininistnitors 

1 1  Provision  of  ndrninimrafivf*  rewards  {o.q.  r^lo^m  time,  pio^uy^ 
for  fiaid-  fnic:  stciff  cinyvsqnd  in  instructional  prnjocts 

12  pirnct  coiniTiunication  bulween  ndministrntion  and  ncademic 
str.ff 

13  Lcirgo  noinber  of  pmpl^  invK^'ved  in  the  docision  niaking 
H5S0ciaU?d  with  thfj  proj^^t 

14  jnitiation  of  projt^cis  by  individual  rnciDbers  of  thu  ucadcrnic 

StCiff 

1 5  Promotioruit  connKiordtions  cj^^en  to  acGdcmic  stcif f  for  work  on 
proioctu 

16  AccKlf^mic  stuff  ff^lt  fiOQd  to  chingo  (n;iEtin(]  ccjui^Q  desiyn  or 
toachiiu]  striitocji^^s 

17  CornmitrmMit  by  acadonuc  stnff  to  irnplurnnnt  and  ovnlualG 
prujccl  dS  dHSigiiad 

18  Student  posiliva  fuudbnck  to  ^.;nciumic  ^taff  on  cnurf^o  design  or 
irnprovJ'mOfU 

19  Opfinnrss  .md  uri-nur.irjnrnr^nl  from  dnpnrimant  cf^airficrson  for 
instmc  t  ic>n."]i  Mnr^^nvoninfU 

20  Coud  porsorKjl  roltitiont]  botwurn  fiiombcrGnf  ucndM/nic  stnff 

21  Expcriunrr:  of  ^Ctidumic  stuff  in  coursn  dufiign,  H  ivijlnprnont  arid 
«veiluonun 

22  Priority  qiv^'n  by  nupport  stiryicun  to  full  najln  fufiflod 
Q  iri3titii!if^n,il  prfijticis 
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UnimpDftsnt  1  2Q3j4  Q  6  7  tmportHrit 

Unirnjjortsnt  1  2  3  4  B  6  7  Irnportarit 

Unlmponunt  1  2  3  4  B  6  7  Imivomant 

Unimportont  1  2  3  4  5  6  7  Important 

Unimportant  1  2  3  4  5  6  7  ImpDrtant 

UnimpDft^nt  1  2  3  4  5  6  7  Importcint 

Unirnpo^tanl  1  2  3  4  5  6  7  Important 

Unlmportan!  1  2  3  4  5  6  7  important 

Unimportant  1  2  3  4  5  6  7  Importont 

Unimportant  1  2  3  4  5  6  7  Important 

Uniniportarit  1  2  3  4  B  6  7  Importont 

rmnt  1  2  3  4  5  6  7  Important 

Unimporiar.t  1  2  3  4  5  6  7  Important 

Unlmpor.ont  1  2  3  4  5  6  7  importont 

Unimportant  1  2  3  4  5  6  7  Important 

Unimportant  1  2  3  4  5  0  7  Important 

Unlmponunt  1  2  3  4  5  6  7  imi^ortsnt 

Unimr»rtant  1  2  3  4  5  6  7  lmnorf?.nt 

Unlmportnnt  1  2  3  4  5  6  7  ImrKsrlont 

Unimportant  1  2  3  4  5  6  7  Important 

UnimporttJfit  1  2  3  4  5  6  7  Imporfnnt 

Unlmrjortant  1  2  3  4  5  6  7  imp^^itrMU 

Ufilmpcjrtfint  1  2  3  4  b  6  7  Import^jni 
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23  Frfiquani  mrrtf       iitiun  bnrwon  ;jciniinintrntion,  fiandumlc 
stfiff,  rcjncu-  nd  nuppor  t  ^l^itf  on  pnigraas  of  prujcct 

24  Prnjo-i  pl.ifit.    ,  ..  livGiy  Uwolmrl  input  from  QCfidnffiic  styff, 
ydfniri!!;! rntiijf),  insiruui lOf in!  dusiyn  DJiinultaf Kb  and  OthCir 

SllppOfl  Sf;rviCOS 

25  Migh  quolily  of  rnaUfficiIs  producjod  for  prDjarjt 

26  Eff icinnny  of  suppof  1  £«»rvicrjs 

27  Posiiivn  cind  suppnrtivu  nuiiuclebof  support  sorvico  porsoiMKd 

28  Good  pr'rf.uruil  rulotic5n3  ii^Mwvu^jn  acfldo-nic  staff  nnd  support 
slfi f  f  on  1  hu  nroject 

29  Ay^ilnbilitv  of  pfofosbiOfKil  consul tnnts  tn  riislst  with  f>dU!;:atianal 
pfiucipir-G  rC'lEiUid  to  thu  Ciif^tgri  ^inrl  uvjli;c!lian  of  projects 

10  Div(;r^i!v  -^^  terfinir?!!  ijijpfK)rt  sorvices,  n  y,  nLjniber  of  iilternniive 
f  fh/dkJ  iivdii.iblu,  CAI ,  r  V,  juciio,  print,  grLiptiic?,  cler ic^t,  t'tc, 

31  Ri/sp^^insfblc  poci'li'-)'!  of  [ir"0|t?ct  dirC'Ctur  in  relation  to 
imA  i  hi  tiOfkJi  li  ifffriri'liy 

32  I nfijffi"if*c)fial  skills  of  prnpM  t  dirnnto'^,  t,g.  sccHssibility,  pulitical 
fiCurih'n.  t.!dn)  l«ador^jlip,  QIC. 

33  C^ljrity  of  prn|Cct  objef  tives 

34  Imtifjiiufi  of  {ifojnnfs  by  proj^^ct  dirac  tor 

36  AiUiiiMivufUris  of  fufjcti inc|/iocjfning  itxliniquus  to  poiGntial 

33  S^Mf-support  (budgtii)  of  projoct  nftyr  iriitial  ciGvolopmcnt 

37  Atirfic;tiv'"ir'S^  of  subjoct  mtincir  to  pOturilinf  studGnts 

38  Sfjvuffil  vic/jd^'!iir:  stdff  fnniribors  tictlvoly  working  on  tfio 
floyulQuff  «:fi ■  of  iho  projoci 

30  Oii^'     two  ac'fldufriic  ^taff  fnernbors  artivoly  working  on  tho 
dovfjIopnMjnt  of  iha  project 

40  Pntonthil  of  iho  prOif^ct  to  gonurate  cducciiinnal  reserirch  findings 

41  A  nurniif^r  oi  pubUcntions  yunortitod  by  tho  prDject 

42  AvAiKitjihty  of  fyrids  for  prn|ftt^t  from  uKtnrnyl  fundinfj  fipennius 

43  f^^ii*  li'l'il^Hity  of  cncf  f^roduct  outsidp  inslituiion 

44  Instriic  tif)nnl  df^sign  5kiMs  fit  prDjf^ct  dirtrctor 

45  N»JrCi5SFiy  to  irnprovu  if i^^rut'f ion  for  3  3(:!f'cific  dinodviintnynd 
stucii-nt  pcipuldtiun 

4G    I  CirfiC  ruirf;h"r  of  ^tucJunt!^  tt;  buHMffl  fiOlTt  projuut 

47  Piuj^v:f  rf?M<M  t!,  rfjnponfi'r  to  currfiot  Euclnl 

48  f  {iiniJl  ^iv.iliiiit icMi  c)f  IIk!  in jtructionr;!  nffcrt!V>,*nufiS  of  tfif^  proiect 

49  niMuCi  p'  f!.nnj|  fukitiL'ns  b''tv#nffn  prcijact  director  diid 
rh  Jiii  iriir.tfntiofi 

50  f'f /'si i(|uHr.  i^jliirr  (jf  U'*' f'f  ^'l^^cf  IhrnufjhnMt  tha  iri«i! itnt inn 

(r)\rm  J.G.  Ileclboro     P.S.  L.iwraoon 


Unlrriportant 

1 

2 

3 

4 

S 

6 

7 

imponiint 

Urihtiportant 

1 

2 

3 

4 

5 

6 

7 

[mporton! 

Unlmpo  riant 

1 

2 

3 

4 

5 

6 

7 

impoftant 

2 

3 

4 

Q 

7 

i  fnfWi  rtnnf 

III  il J\J  1  i^l  i  1 

Unirnportorit 

1 

2 

3 

4 

5 

6 

7 

Important 

Unimportant 

1 

2 

3 

A 

5 

B 

7 

Impoftsnt 

UninipD  riant 

1 

2 

3 

4 

5 

6 

7 

Important 

Unlmportsnt 

1 

2 

3 

4 

6 

6 

7 

Important 

Unjmp/Ortant 

1 

2 

3 

4 

5 

6 

7 

Irnportant 

Uniffi  portent 

1 

2 

3 

4 

D 

6 

7 

Important 

Un  iiTi  p^rtiint 

2 

3 

4 

5 

6 

7 

1  im^Q  rt  a  n  t 

UnirnpOrtant 

1 

2 

3 

4 

5 

6 

7 

important 

IJnirripartant 

1 

2 

3 

4 

5 

6 

7 

Important 

1  Irl  iril  i=ViPf  An  1 

w  1 !  Mi'      '     r  ■  1 

^ 

2 

3 

4 

5 

g 

7 

i  fflrV^  rfnnt 

1  1  1  ll:JV'  1    Itll  1  L 

Unimptfr  ^ant 

1 

2 

3 

4 

5 

6 

7 

Important 

Uriimportfint 

1 

2 

3 

4 

5 

6 

7 

Irnpo  riant 

Unimportafit 

1 

2 

*^ 

4 

5 

6 

7 

Importunt 

1^  1 1  i  1 1 '  ^^^^1  ^yi  > 

2 

3 

4 

g 

7 

2 

3 

4 

G 

7 

1  IT  i^UI  &U  1  % 

tJn  ifTi^ortsiit 

-{ 

2 

3 

4 

G 

7 

t  f¥i  Rrtrf  an  t 

•  ii  1           ts!  1 1 

V#  [  I  1  i  M  ^  f  V  P  t  lH  1  I 

i 

2 

3 

4 

5 

0 

7 

1  1 1  1 1  nJ  1  I  0  i  <  I 

Ur^iirtportgnt 

1 

2 

3 

4 

5 

6 

y 

important 

1 

2 

3 

4 

0 

6 

7 

Irnportant 

Unirnp^rtsn  t 

1 

2 

3 

4 

5 

e 

7 

1  i^i'^orton* 

2 

3 

4 

5 

6 

7 

1  nipori  snt 

Unimportant 

1 

2 

3 

4 

5 

6 

7 

Important 

Un[fTipoit<int 

1 

2 

3 

4 

5 

6 

7 

Ufiimpoftsnt 

1 

2 

3 

4 

5 

Figure  1 


Path  Diagrams  for  SuccossfLil  Instructional  Projects 
{Lawrason  &  Hedberg,  1978) 


(a)  Independent 
Variables 
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(BANK) 
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1     Tho  words  in  italics  and  in  parentheses  are  shorthand  forms  used  in  the 
statistical  program. 
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Figure    2     Path  diagrams  for  ^;uccossful  instructional  projects  baaed  upon 
an  Austral ian  sample. 
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Tahia    1        Tho  Naturo  of  the  Projocts 


Varidble 


Category 


U.S.  Frequcficies 
N-  38  ^ 


Austral ian 
f  rLfquenciGS 
27  ^ 


PSIZE 

The  siKu  op 
thQ  project 


1 

Smaner  than  a  coursa 

2 

6 

2 

One  coLirsr' 

5 

7 

3 

Moro  than  ono  course 

31 

10 

4 

Gonnral  project  related 
to  teaching 

0 

3 

RANK 

The  QcadcMiiic  rank 
of  tho  res pond nnt 


1  Senior  adnti ni strator 

2  Depcirtniental  Chair 

3  Tenured  fciculty 

4  Nnn-tneured  faculty 

5  Support  staff 


7 

4 
21 
5 
1 


3 

4 

17 
2 
1 


Pdrt  respondent 
played  in  the 
project  team 


1  ^ '  ^  .:t  director 

2  Instructional  designer 

3  General  support 

4  Not  a  member 


3 

11 

o 


5 
3 
1 


PINIT 

Raiik  of  person 
wlio  initiaLcjd 
the  project 


1  Admi  ni  strator 

2  Faculty 

3  S  ipport  s  taf  f 


22  3 
15  21 
1  2 


If  the  totals  are  less  than  38  or  27  respectively ^  the  remaining  responses 
are  missing. 
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Tablfi  2      Cpiiiposition  of  F  '':i:ur&  contributing  to  itmtnjctional  innovation 
success.  (Dased  upon  Australian  Sample.  N---27) 


ORGSLJI' 
STATUS 
CLIMAT 

R'PFRT 


Itam  5,  12,  23,  26,  27,  28,  49 

Iteis  10,  lb,  ?Ji,  30,  32,  39,  40,  41,  43,  5C 

Items  3,  4,  19,  20,  35,  37,  42* 

Items  17,  21,  33,  34,  rt^ 


Item  42  ii,  reverse  scored. 


Table  4       Compai'i'on  of  Dependent  Measures 


Variable 

SUCC 
rUND 
DIR 
STUD* 


Mean  U.S.  data 
(N=  38) 

5.8 
5.3 
4.5 
5.3 


Mean  Austrolian 
data    (N=  27) 


5.4 


5.4 


4.B 


t-value 

-1.30 
0.34 
1.39 

-1.31 


.19 
.74 
,17 

,20 


*  This  ciuestlon  war.  onritod  by  iiiany  respondents  -  satriples  were  28  and  20 
rospGctively  .for  tlris  cjucstion  unly 
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Table  3    Siinple  correlations  betwsefi  variables  in  the  Australian  sanple 


mi  m 
m  M 


pm 


m  ^  QR3SU?   STATUS  CLIli     EIPE^T  SOCC 


m 


.03  -.05 


.07 


.0/ 


.01 


.2/  ',1 

!/  M 

.40 

CLIli 

-.05 

19  .11 

.15 

.07 

Uf  \  i  .„  i  i  1 

-JO 

il  =.02 

.05 

.38 

SjCC 

.29  .[ 

)5  ,04 

.12 

.26 

.32  .37 

FlInO 

^.12 

.13  J 

i  .06 

-.21 

".07 

-.10      ,09  M 

m 

3^  .1 

=  .10 

-.15 

.00     -.02  .00 

STOO 

-.0/ 

.3  -J4 

.34 

=.15 

.29      .10  .23 

.65 


23     '.n  .08 


EFFECT  ON  INTENDED  AND  INCIDENTAL  LMRNING 
FROM  THE  USE  OF  LEARNING  OBJECTIVES 
WITH  AN  AUDIOVISUAL  PRESENTATION 


A  papar  presented  to  the  Research  and  Theory 
Division  of  the  1979  AECT  Convention  by 
Robert  Main,  Associate  Professor  at  the  Center 
for  Information  an^^  Coimnunication  Studies, 
California  Ftate  \      versity,  Chico ,  California, 
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ABSTRACT 


Thic  paper  reports  a  controlled  field  experiment 
conducted  to  deterinine  the  effect  and  interaction  of  five 
independent  varibles  with  an  audiovisual  slide^tape  piogram- 
presence  of  learning  objectiA/es,  location  of  learning  objectives 
type  of  knowledge^   sex  of  learner^  and  retention  of  learning. 
Participants  were  university  students  in  a  general  studies 
course.     Test  score? measuring  intended  and  incidental  know^ 
ledge  were  the  dependent  variables.   The  major  finding 
supported  Ausubel's  subsumption  theory  in  that  use  of 
Ic?.  rning  objectives  as  an  advance  organizer  presented  before 
thu  slioe-tape  programs  signif icantiv  increased  intended 
learning.     No  significant  loss  was  taiind  for  incidental  learning 
Indeed,   from  the  direction  of  the  data,   the  tendency  was  for 
the  learning  objectives  to  improve  rather  tha^i  lessen  incident-^ 
al  learning.     The  use  of  the  leatning  objectives  before  the 
presentation  had  no  significant  effects  on  the  affective 
variables  measured  and  did  not  detract  from  the  students - 
evaluation  of  the  program.     To  the  contrary,   the  trend  of  the 
data  indicated  students  receiving  learning  objectives  prior 
to  the  presentation  liked  the  program  more  than  wlien  no  learn- 
ing objectifies  were  used. 


Introduction  and  Background 

The  non  rint  media  are  becoming  Increasingly  important  as  educational 
tools  for  r\mri.  ,n  schools.    The  increase  in  availability  of  educational 
media  material s  for  school  use  has  been  dramatic.    Twenty-five  years  ago, 
perhaps  25,000  nonprint  instructional  programs  might  have  been  available  to 

the  elementary  and  secondary  schools  of  this  country,^    Today,  the  number  of 

2 

titles  1s  Rearing  500^000.     Because  instructional  media  of  many  kinds  are 
important  in  structiiring  what  goes  on  in  classrooms  and  what  students  d£  to 
learnj  consideration  of  ways  to  improve  their  use  1^  important. 

It  would  be  beneficial  for  teachers  to  have  practical  methods  for  in- 
fluencing the  efficiency  and  effectiveness  of  audiovisual  materials  selected 
for  curriculum  support.    There  is  little  In  the  Uterfiture  to  guide  the 
teacher  in  the  application  of  Instructional  theory  to  the  nonprint  media. 
The  empirical  evidence  regarding  learning  objectives,  advance  organizers,  and 
mathemagenic  activities  to  facilitate  learning  is  largely  concerned  with 
print  materials. 

Printed  curriculum  i/iater'^'al s  allow  the  learner  a  wide  latitude  in  use-- 
browsing^  skipping,  review,  repetition.    Through  selection,  pacing,  and 
sequencing,  the  learner  may  Individualize  the  material  to  his  particular 
needs  or  preference.    Audiovisual  materials,  on  the  other  hand,  are  commonly 
fixed  pace,  fixed  sfjquence  presentations  with  content  selected  and  organized 
for  use  with  as  wide  an  audience  as  possible. 


The  purpose  of  this  study  was  to  investigate  the  effects  of  the  use  of 
learning  objectives  with  an  audtovisual  presentation  on  the  intended  (obj ac- 
tive relevant)  and  incidental  (nQri-objc^ctl ve  relevantj  learning  outconies. 
This  study  was  designed  to  provide  evidence  regar:rrig  the  facilitatlve  effects 
of  using  learning  objectives  as  an  organizer  with  a  fixed-pace,  fixed -format, 
non-print  medium, 

theoretical  Founda  t1 ons 
The  primary  theoretical  base  for  this  study  Is  anchored  in  the  cognitive 

construct  Instructional  theories  and  the  concept  of  external  organizers  to 
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facilitate  learning  as  developed  by  Ausubel.     The  use  of  learning  objectives 

4 

as  organuinq  tools  1s  derived  from  Gagne*s  principles,    and  the  concept  of 

modifying  learning  activity  by  differential  placement  of  the  organizing  device 

g 

is  developed  from  Rothkopf' s^  hypotheses  of  inathemagenic  behavior. 

In  opposition  to  the  more  behaviorally  oriented  theories  dominated  by 
Skinnf-ri  '  the  cognitive  construct  instructional  theorihts  attribute  a  greater 
degree  r:    vtonomy  and  initiative  to  the  learner.    Typically^  they  emphasize 
the  student's  internal  representations  or  cognitive  constructs  as  the  really 
important  concern  in  education. 

Advance  Organizers 

Of  particular  interest  to  this  study  is  the  emphasis  on  organization  as  a 
critical  component  of  the  learning  process  as  espoused  by  David  P.  Ausubel  and 

others.^   Ausubel  has  articulated  these  ideas  into  a  theory  of  meaningful 
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verbal  learning  which  he  calls  the  subsumntion  theory. 

Briefly,  the  theory  holds  that  meaninu'.     verbal  learning  can  be  incor- 
porated into  the  learner's  cQgnitlve  structure  and  there  be  ret^^'ned  more 
accessiblei  and  for  longer  periods  of  tlj.ifij  if  the  cognitivr  structure  1nn:ially 
contains  discerniblei  relevant,  and  related  b^\s  of  information  (subsi'mars) 


which  the  learner  can  associate  with  new  learning.    With  the  passage  of  time, 
tfie  Incorporated  learning  will  undergo  successive  reformations  (obliterati ve 
subsumptlon)  as  the  learner  attempts  to  condense  and  hierarchically  arrange 
added  learning, 

Ausubel^  has  supported  his  theory  with  experlfnental  evidence  which 
indicates  that  (1)  those  students  who  are  known  to  have  relevant  subsumer?. 
will  remember  more  of  the  subsequent  learning  for  a  longer  period  v    >  Mne 
compared  to  those  who  do  not;  and  (2)  low  achievers  exhibit  great  .pc^ases 
In  achievement  with  the  aid  of  subsuniers  than  high  achievers.    Ausu  ■  :  attri- 
butes this  effect  to  the  apparent  lack  of  organization  in  the  cognitive 
structure  of  the  low  achievers.    Externally  imposed  "advance  organiiers" 
provide  the  necessary  organization  in  low  achievers  but  are  of  little  aid  to 
high  achievers  who  impose  their  own  organization  on  unstructured  material* 

Accordini  to  Ausubel  and  Robinson,  optimal  utilization  of  the  principles 
of  progressive  differentiation  and  integrative  reconciliation  can  be  achieved 
by  the  " , . , suppl ementary  availability  of  a  hierarchical  series  of  advance 
'organizers. ' True  organizer;,  in  Ausubel  and  Robinson's  view^  are  not  to 
be  confused  with  ordinary  Introductory  overviews  which  are  typically  written 
at  the  same  level  of  abstraction,  generality,  and  incl usiveness  as  the  learn- 
ing material  and  achieve  their  effect  largely  through  repetition,  condensa- 
tion, selective  emphasis  on  central  concepts,  and  pre-fami 1 larization  of  the 
learner  with  certain  key  words.    They  cons1der"f?d  summaries  as  cofnparable  to 
overviewst  but  probably  less  effective  because  their  influence  on  cognitive 
structure  1s  retroactive  rather  than  proactive  ^^alative  to  the  learning  task. 
They  felt  overviews  and  summaries  "insofar  as  they  imply  to  some  learners  that 
the  mater »nl  th:uy  dc  not  include  is  relatively  superfluous  .  ,  .  may  promote 
/  -:nt-:t  ■  r  «nd  failure  to  study  or  review  much  significant  subject  matter."  ^ 


Ausubel  and  Robinson  also  felt  advance  organfzers  had  ■certain  Inherent 

advantages"  over  various  kinds  of  Intramaterial  organization  (adjunct  organ'z- 
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Ing  aids  within  the  body  of  the  material).       They  reasoned  that  ex^^stina 

m  - 

-n  \mtrs  in  cognittve  str  enure  lacked  both  particularized  relevance  for  the 
new  material  (since  the  '   ■        could  not  possibly  anticipate  its  precise 
nature)  as  well  as  thr     priistic  tted  knowledge  of  subject  matter  and  pedagogy 
available  to  expert  p  'ogra  ars. 

Behavioral  Objectives 

The  avenue  to  be  explored  by  this  study  was  the  extent  that  learning 

objectives  could  be  used  as  organizing  devices  for  audiovisual  presentations, 

Gagne  is  a  principal  proponent  of  the  use  of  behavioral  objectives  as  faci11» 

tators  of  learning.    He  states: 

Defining  objectives  for  the  student  is  an  inadequately  exploited  educa- 
tional technique  .  .  .  .    (U)nless  students  know  what  the  objectives 
are,  they  are  likely  to  resort  to  memorization  and  mechanical  com- 
pletion of  the  exercises  In  textbooks  or  workbooks  rdther  than 
carryinn  out  relevant  sorts  of  learning  activities.    When  one  tells 
the  stu^  mt  what  he  ■    expected  to  do  after  he  learns,  he  is  not 
"giving  ..i^n  the  ans'^    . Rather  1t  is  providing  him  with  a  goa] 
which  lit-  ^^ifu^eif  can  Lie  to  organize  his  own  learning  activities^ 

The  use  o-  onjectives  as  organizational  devices  Is  also  sup'^orted  by 

Garrison  and  Magoon  who  state,  "Organization  is  the  process  which  converts  the 

raw  materials  of  learning  into  a  constructive  approach  toward  meeMng  an 
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jucatlonal  objective."      They  emphasize  the  importance  of  objectives  as  a 

learning  aid  by  the  students  in  Interacting  with  the  instructional  material  by 

stating:    "Learning  experiences  should  correlate  closely  with  the  established 

objectives  of  which  the  learner  has  been  fully  apprised. "^^ 

Ellis  also  links  objectives  and  organization  as  important  conditions  to 

learning  from  the  student's  viewpoint. 


A  major  feature  of  good  study  habits  is  that  you  understand  the 
principle  objectiv_es  or  goal  s  of  whatever  H  j£  ^£u  are  study ina  .  . 
•  .    (K)eep^  In  mincj  that  you  are  trying  to  get  a  good  ^'mental 
roadmap"  of  the  inaterUl  before  you  study  it  in  greater  detail, 

.  ,  *    If  structure  is  not  immediately  evident  in  the  materiaU 
then  you  must  seek  structure  or  create  some  kind  of  useful  system 
for  organizing  the  materipls.    This  is  simply  to  say  that  y^g  must 
impose  some  kind  of  or^gani zation  upon  the  material  c  .  •  .  . 

Mathemagenic  Behavior 

Another  theoretical  area  relevant  to  this  study  is  the  concept  of  mathe- 
magenic behavior  developed  by  Rothkopf  and  his  associates,    Rothkopf  asserts 
that  the  effect  of  an  instructional  document  can  be  affected  to  some  degree  by 
controlling  the  behaviors  of  the  learner  in  relation  to  the  document.  Since 
these  behaviors  affect  learning,  Rothkopf  has  called  them  mathemagenic 
behaviors*  a  word  of  Greek  origin  which  means  behaviors  "v/hich  give  birth  tc 
learning, "^^    He  postulates  that  there  Is  a  variety  of  activities  in  which  th.; 
instructor  or  learner  can  engage,  which  can  modify,  and  to  a  degree  control, 
mathemagenic  behaviors* 

Most  of  Rothkopf's  rese'^^f^^h  'nvolves  th^^  use  of  questions  inserted  in 
written  materials  as  a  meani:  of  r^^  J  ir^nrg  r? pit^Hmagenic  behavior  to  increase 
learning.    There  have  been  a  number  wv  studies  by  Rothkopf  a?id  others  to 
support  his  hypothesis.    Several  variables  have  been  examined  in  the  experi- 
ments but  they  all  have  one  variable  in  -ommon--question  location.    In  general, 
Rothkopf's  studies  indicated  that  grouped  pre-questions  and  adjunct  pre-  and 
post'-quos t Ions  resulted  in  nr^eater  learning  of  question'-speciflc  relevant 
kriowled^^j  than  from  either  a  no-question  or  grouped  post-question  treatment. 
He  also  found  more  learning  of  non-relevant  question  information  by  the  adjunct 
post-question  treatment  only.    The  frequency  of  adjunct  post-questions  also 
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tended  to  aid  learning. 


Rothkopf  suygests  that  pre-questioni  serve  a  cuf  ng  function  of  informing 
and  dlrectfng  the  learner  regarding  important  content.    Post-questions  do  not 
affect  salectlve  attendance  to  the  material  but  rather  force  a  mental  review 
of  all  material  (both  Intended  and  incidental  knowledge)  in  order  to  answer 
the  question. 

Intended  and  Incidental  Learning 

The  facilitatlve  effects  of  using  learning  objectiv'es  as  directions  to 
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learn  relevant  prose  text  material  has  been  well  documented.       The  .^elrU'^t^ 
effects  of  objectives  with  respect  to  incidental  (non-object1 vt  /ant; 
learning  are  less  clear*    Several  studies  have  reported  greater  Incidental 

learning  for  groups  receiving  objectives  (or  questions)  than  for  non-objective 

21  - 
control  groups.       Other  investigations  have  shown  a  decrease  1n  learning 

performance  when  Instructional  objectives  are  presented  before  the  learning 

22 

materials.    Duchastel  and  Brov;nj  ^  for  example^  found  that  subjects  v^ho  were 
given  objectives  performed  somewhat  more  poorly  on  incidental  material  than  a 
control  groLip  who  received  no  objectives*    The  reason  for  the  difference 
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between  the  results  of  these  experiments  is  not  clear.    Kaplan  and  Simons 

tosted  several  hypotheses  concerning  the  reasons-  for  the  observed  increases  in 

learning  found  In  these  studies.    Specif Ically,  they  investigated  the  effects 

of  the  location  of  the  Instructlona i  objectives  with  respect  to  the  learning 

fnaterial  (before  vs,  after).    Their  principal  finding  was  that  performance  un 

Information  relevant  to  an  objective  was  relatively  high  whether  the  coj-ic- 

tives  were  presented  before  or  after  tne  text.    Perfarniance  on  Incidencil 

material  was  greater  for  objectives  located  after  the  text  than  before  the 
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text.    These  resuHs  are  generally  supported  1n  other  studies  by  Brunlng^ 
25  26 

Frase»"    and  Rothkopf     which  show  that  overall  retention  tends  to  be  greater 
whe    questions  are  located  after  rather  than  before  text  segments* 
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The  c^Gnerai  concUision  is  thdt  ohject^'/fty  o^^  questions  presented  after 
the  learning  inaterlals  cannot  be  used  as  orienting  stimuli  which  permit  the 
learner  to  selectively  attend  to  specific  information  in  the  text. 

Virtually  all  of  the  research  concerning  behavioral  objectives  as  organli- 

ing  devices  hes  been  concerned  with  printed  material    Lumsdaine,  May  and 
27 

Hadsell     studied  the  effects  on  learning  of  supplementary  instruction  pre= 
sented  before  or  after  a  film.    While  they  did  not  use  "learning  objectives" 
they  did  employ  a  variety  of  techniques  to  direct  attention  to  specific  Infonna- 
tion  1n  the  filni  content.    Although  intended  learning  did  Increase^  the  effects 

were  sniall  and  the  results  Inconclusive,  particularly  with  regard  to  any 
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effect  to  inc  '  sntal  learning,    Dayton     conducted  a  recent  Investigation  upon 
the  effects  of  inserted  questions  in  a  slide- tape  presentation.    He  found  a 
significant  increase  in  the  amount  of  question-relevant  learning  but  the 
effects  on  incMental  learning  were  ambiguous. 

In  light  of  these  studies ^  the  fr "lowing  questions  may  be  asked: 

1.  Does  the  insertion  of  learning  objectives  in  slide-tape  presenta- 
tions result  in  greater  learning  than  would  occur  as  a  result  of  a 
slide-tape  presentation  only? 

2.  What  effect  does  the  location  of  the  learninc;  objectivei:  in  respect 

to  the  presentation  have  on  student  learning  from  slide- tape  materials? 

3.  What  are  the  differential  effects  of  the  placement  of  learning 
objectives  upon  the  intended  and  Incidental  learning  by  the  student 
from  slide-tape  presentations,  1.e*»  is  the  effect  generally  facili- 
tating (results  in  greater  learning  of  all  content)  or  is  it  due 
only  to  review  (results  in  greater  learning  of  material  relevant 
only  to  the  learning  objectives)? 

Answers  to  these  questions  should  provide  useful  information  for  people 
Involved  in  the  use  of  media  as  an  instructional  tool.    Most  of  the  audiovisual 


presentations  avanable  have  not  been  designed  with  specific  terrriinal 
behavior  patterns  in  rnlnd.    Frequently  the  materials  used  1ri  Instruction 
were  not  even  intended  for  educational  purposes.    It  would  be  of  yalue 
to  tedcherSi  to  producers  of  audiovisual  programs,  and  to  curriculuni 
developers  to  know  the  effects  the  introduction  of  learning  objectives 
to  the  program  will  have  on  facilitating  the  intended  knowledge  acquisition. 
The  addition  of  learning  objectives  would  add  very  little  to  initial  produc- 
tion costs  and  could  be  added  for  minifndl  costs  to  existing  Inventories  of 
slide  and  video  progranis*    It  v/ould  be  more  costly  to  insert  learning  objec- 
tives 1n  motion  picture  prIntSi  but  suppleniental  projection  of  objectives  by 
35  mm  projectors  would  be  practical  when  using  films. 

This  study  niay  be  of  equal  interest  in  defining  the  effect  of  the  irsertion 
of  learning  objectives  on  the  acquisition  of  incidental  knowledge.    There  has 
been  a  great  rush  to  increase  the  use  of  behavioral  objectives  and  their 
efficacy  in  improving  learning  outcomes  is  well  documented.    Less  attention 
has  been  paid  to  their  potential  in  terms  of  their  effects  on  Incidental 
learning.-  The  results  of  this  study  should  serve  not  only  as  a  guide  to 
benefits  of  inserting  learning  objectives  in  audiovisual  presentations,  but 
also  as  an  indication  of  possible  side  effects  of  loss  in  incidental  knowledge. 
This  caution  may  be  of  significance  in  certain  situations  where  the  emphasis 
is  to  expand  abnit1es  to  synthesize  material  rather  than  focus  on  information 
transfer.    In  addition,  the  answers  to  these  questions  would  have  Ifripl  icatlons 
in  determining  the  validity  of  the  propositions  by  Ausubel  and  Gagne  and  by 
expanding  the  implications  to  Include  the  use  of  learning  objectives  as  organ- 
isers for  audiovisual  materials. 


study  Design 

A  controned  field  experifnent  was  conducted  to  measure  the  Intended  and 
Incidental  learning  outcomes  resulting  from  the  use  of  learning  objectives 
with  a  selected  audioylsual  presentation. 

The  experiment  was  designed  to  determine  the  effect  and  Interaction  of 
five  independent  variables  with  an  audiovisual  slide-tape  program:  presence 
of  learning  objectives,  location  of  learning  objectives^  type  of  knowledge, 
sex  of  learners  and  retention  of  learning.    A  4x2x2  factorial  design  repli- 
cated for  retention  of  learning  was  selected  for  the  analysis.  Student 
learning  of  intended  knowledge  and  incidental  knowlodge,  as  measured  by  paper 
and  pencil  tests,  were  the  dependent  variables.    The  design  model  1s  at  Figure 

A  20  minute  commercially  produced  educational  slide-tape  presentation 
concerning  advances  in  mass  media  and  communications  was  used  as  the  stimu- 
lus'^  The  participants  were  students  enrolled  in  an  introductory  mass 
communication  course  at  California  State  University,  Chico,  in  the  spring 
semesteri  1977.    The  coursa  v/as  required  for  all  comriunication  majors  and  wasi 
in  addition,  a  service  course  for  the  University's  general  studies  prograin. 
The  course  content  was  heavily  mediated  with  almost  every  class  meeting  using 
a  film,  videotape,  audiotape,  or  multlniedia  presentation  to  support  the  in- 
struction.   Students  in  the  class  were  divided  into  three  treatment  groups  and 
one  control  group  by  a  random  saftipling  procedure  stratified  for  sex* 

The  treatment  factor  was  the  presence  or  absence  of  learning  objectives. 
Learning  objectives  were  used  with  the  slide-tape  presentation  for  experimental 
groups  1,  2,  and  3,    No  learning  objectives  were  available  for  experimental 
group  4  which  constituted  a  control  group. 

The  learning  objectives  for  the  course  content  contained  1n  the  slide- 
tape  treatment  material  were  established  by  interviewing  the  professor  of  the 


SEX 


GROUPED  AS 
ADVANCE  ORGANIZER 


GROUPED  AS 
POST  ORGANIZER 

LEARNING  OBJECTIVES 

INTERSPERSED  AS 
ADJUNCT  ORGANIZER 

NO  LEARNING 
OBJECTIVES  SPECIFIED 
(CONTROL) 


IMMEDIATE 
POST  TEST 


DELAYED 
POST  TEST 


INTENDED  INCIDENTAL 


KNOWIEDGE  CATEGORY 


Ftiur*  1 
EMptf Inane  Hedil 


crt»tn«nc  (prsianet  and  lesaetan  et  lairntnt  ebj4sliv«i}  u4  Ms  <Mla- 
fataala),    Tha  dalaytd  InlandN  and  ineidintal  kn€irl«4tt  poatttae  seeraa 
tTQ  lubjtceid  ss  Idintlsil,  analysti  of  verlaMa  JtUavina  «MH'^M>n  of 
data,    The  Isnadlflta  poaceiai  teoria  ara  trtiC*^  la  i  isiMrtct  but 
idantical  aiiparlaantt 


ERIC 


278 


course,  taking  her  educQtional  alms  and  specifying  the  terminal  behavior 
requirements.    The  Initial  objectivis  were  then  5ubrii1tted  to  five  professors 
who  had  previousl)^  taught  the  introductory  mass  cornniunlcation  course  and  were 
thus  familiar  with  the  educational  aims  of  the  particular  block  of  instruc- 
tioHi  but  were  not  familiar  with  the  sUde-tape  programi    Each  professor  was 
asked  to  rank-order  the  learning  objectives*  modify  as  appropriate^  and  add  or 
delete  objectives  if  necessary.    In  addition,  each  professor  was  asked  to 
furnish  three  questions  designed  to  measure  by  paper-pencil  method  the  in» 
tended  behavior  specified  by  each  objective.     From  this  modified  list  of 
learning  objectives >  four  receiving  consensual  judgment  as  being  appropriate 
were  selecteri.    Each  learning  objective  was  then  reproduced  on  a  35  mm  slide* 

The  location  of  treatment  factor  was  the  application  of  the  learning 
objectives  inserted  as  a  group  before  the  presentation  (advance  organizer) , 
Inserted  as  a  group  after  the  presentation  (post  organizer),  and  interspersed 
within  the  presentation  (adjunct  organizers).    The  audio  track  for  the  pre- 
sentation was  not  changed  except  to  add  10  seconds  of  silence  for  each  behav^ 
ioral  objective  slide. 

The  type  of  knowledge  factor  constituted  the  cognitive  level  of  infonna- 
tion  relevant  to  the  learning  objectives  (intended  knowledge)  and  the  cogni- 
tive level  of  information  non-relevant  to  the  learning  objectives  (incidental 
knowledge) * 

The  Intended  and  Incidental  knowledge  content  of  the  slide- tape  presenta- 
tion was  determined  by  a  content  analysis,    The  program  was  presented  to  a 
graduate  class  in  instructional  theory  and  design.    Copies  of  the  script  and 
the  learning  objectives  ware  furnished  each  student,  who  was  then  asked  to 
list  the  major  points  of  Information  contained  1n  the  presentation  that  did 
not  directly  contribute  to  the  learning  objectives,    This  body  of  information 
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was  designated  as  incidental  knQwledge  and  test  questions  were  developed  to 
measure  the  cognitive  level  of  this  Incidental  knowledge.    All  other  informa- 
tion was  considered  intended  or  objective-relevant  knowledge. 

The  Instrument  used  to  ineasLire  knowledge  1n  the  experiment  was  a  pencil 
and  paper  test  with  three  parts,    Part  one  nieasured  Intended  learning  and 
consisted  of  10  Items  provided  by  the  learning  objectives  development  panel  as 
dl£crete  measurements  of  intended  learning  behavior. 

Part  tv/o  of  the  instrument  measured  Incidental  learning  and  consisted  of 
12  items  developed  during  the  content  analysis  by  the  instructional  theory  and 
design  class. 

Part  three  of  the  instrument  measured  affective  variables  concerning  the 
subject  matter  and  the  presentation.    It  consisted  of  10  items  each  with  a 
series  of  statements  arranged  in  a  Likert  type  semantic  differential  scale 
ranging  In  value  from  one  through  five.    The  variables  nieasured  were:  (1) 
optlrnlsm  concerning  the  contribution  to  society  of  communication  technologya 
(2)  the  Importance  of  coninunication  media  to  society,  (3)  relevance  of  material 
to  the  classs  (4)  bias  or  objectivity  of  the  presentation^  (S)  Interest  in  the 
subject i  and  (6)  evaluation  of  the  presentation.    The  evaluation  of  the  pre- 
sentation was  further  defined  by  five  categories: 

1,    Overall  evaluation 

2»  Organization 

3,  Length 

4,  Qual ity  of  visual s 

5,  Format 

The  test  items  for  nieasuring  the  affective  domain  were  developed  by  the 
researcher  and  were  not  subjected  to  a  validity  or  reliability  test. 


A  pilot  study  to  establish  the  validity  of  the  test  instruments  was 
conducted  by  presenting  the  audiovisual  presentation  to  33  students  enrolled 
in  an  undergraduate  course  in  broadcast  nianagement ,    Mo  behavioral  objectives 
were  given  to  the  students.    The  test  was  administered  immediately  following 
the  presentation. 

The  results  of  the  pilot  test  indicated  the  internal  consistency  of  the 

test  instrument  was  acceptable  with  a  Kuder-Rlchardson  reliability  of  .75. 
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This  was  considered  satisfactory  ^or  the  experiment's  purpose. 

No  analysis  was  made  of  the  affective  measurement  items  other  than  inspec- 
tion to  deterrtilne  that  variance  was  present.    The  learning  retention  factor 
was  the  cognitive  level  of  intentional  and  incidental  information  measured 
immediately  following  the  presentation  and  measured  again  after  a  14  day  time 
lapse. 

The  experiment  was  conducted  on  May  3,  1977  during  the  normally  scheduled 
class  meeting  time.    Viewing  and  listening  factors  were  carefully  controlled 
for  each  group.    The  subjects  were  directed  by  name  to  four  separate  and 
identical  classrooms  equipped  with  Identical  35  mm  Kodak  slide  projectors 
synchronized  by  identical  Wollensak  audiotape  playback  machines*    Table  1  1s 
the  schedule  of  presentation  treatments.    The  subjects  observed  the  slide- tape 
presentations  without  additional  instruction,    Immediately  following  the 
presentation 4  a  graduate  assistant  administered  the  examination.    All  questions 
were  contained  In  a  test  booklet  and  the  students  recorded  their  responses  on 
a  standard  machine  graded  answer  sheet.    This  was  the  normal  examination 
procecfure  employed  for  the  class  and  consequently  the  students  were  famniar 
with  the  procedure. 


Table  1 
Presentation  of  Treatments 


Group 


Schedul  e 


Learning  objectives  presented  as  a  group  1n  advance  of  slide- 
tape  presentation 


2 


Learning  objectives  presented  as  a  group  at  conclusion  of 
slide-tape  presentation 


3 


Learning  objectives  presented  Individually  throughout  presen- 
tation immed iately  preceding  relevant  portion  of  slide-tape 
presentation 


4 


Slide-tape  presentation  only;  no  learning  objectives  provided 


In  the  follow-up  test  conducted  on  May  17,  1977^  the  subjects  were  given 
Identical  test  booklets  in  their  regular  classroofn,  answered  the  questions, 
and  turned  1n  the  booklets  at  the  end  often  minutes.    Part  III  (affective 
measurement)  was  delated  from  the  fonow-up  test  booklets. 

To  avoid  as  much  as  possible  the  effects  of  learning  from  the  testi  the 
test  booklets  were  conected  after  each  post-test,  and  the  results  of  the 
irnnediate  post-test  were  not  announced  to  the  participants. 


The  raw  test  data  collected  on  mch  student  participating  1n  the  experi- 
meiit  consisted  of  108  matched  immediate  and  delayed  post-test  scores^  equally 
divided  into  three  treatment  groups  and  a  control  group.    The  number  and 
source  of  raw  test  data  scores  collected  on  each  student  and  used  for  the 
analysis  are  llsteJ  in  Table  2. 

Additional  data  Items  on  each  participant  were  collected  from  student 
records  maintained  by  the  class  instructor.    These  were  sex,  class  level  and 
grade  point  average  (GPA)  for  the  course.    A  ch1-square  statistic  was  computed 
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Table  2 

Data  Scores  Available  for  Analysis 


n       f4  .  Number  and  Source  of  Scores 

Uata  Item 

Imfiiediate  Post-test         Delayed  Post-test 


Intended  learning  108  108 

Treatment  1  27  27 

Treatnient  2  27  27 

Treatment  3  27  27 

Control  (no  treatment)  27  27 

Incidental  learning  108  108 

Treatnient  1  27  27 

Treatiiient  2  27  27 

Treatment  3  27  27 

Control  (no  treatment)  27  27 

Affective  domain 

Optimism  108 

Importance  108 

Bias  108 

Evaluation  of  presentation 

Overall  108 

Relevance  108 

Organisation  108 

Length  108 

Visual  quality  108 

Fomiat  108 

Interest  108 


to  test  the  ''goodness  of  fit^'  of  the  sample  distributions  across  each  of  these 
characteristics.    The  results  indicated  the  sample  groups  were  relatively 
homogeneous  for  the  characteristics  observed  with  no  significant  differences 
existing  between  distributions  across  sexs  class  level  or  grade  point  average* 

The  mean  score  of  each  treatnient  group  for  the  intended  and  Incidental 
knowledge  measurements  are  displayed  in  Table  3,  A  graphic  presentation  of 
the  dependent  variables  1n  percentage  figures  is  shown  in  Figure  2, 
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Figure  2 


Pattern  of  Dependent  Variable  Scores  by 
Knowlidgt  Type  and  EKperimcntiil  droup 


^Inttndcd  knowledgi  percencage  icore. 
bincldintal  knowlidge  percentage  Hcore* 


Table  3 

Mean  Scores  of  Test  Responses 


Knowledoe 
Type 

Advance 
Org. 

Treatment  Group 
Post  AdjiJnct 
Org.  Org, 

Cnn t  ro 1 

Delayed 

Mean 

—    ^^-^  ...^^ 
7.65 

6.77 

6.34 

6.29 

Intended 

S.D. 

2.4 

2.4 

2.4 

2.5 

Immediate 

Mean 

8.  OS 

7.52 

7.02 

5.95 

Intended 

S,D, 

2,3 

2,3 

2.3 

2.2 

Delayed 

Mean 

6.89 

6.85 

5.59 

6.56 

Incidental 

S.D, 

1.9 

2.3 

2.2 

2.0 

Ininedlate 

Mean 

7.70 

7.30 

6,30 

6.96 

Incidental 

S.D, 

2.1 

2,3 

2.3 

1.8 

The  Hypotheses  Tests 
There  were  three  research  hypotheses  applied  to  the  study.    They  ware 
that  under  the  conditions  of  the  experliTient : 

L     There  is  no  significant  difference  1n  cognitive  learning  outcoines  of 
objective  relevant  (intended)  knowledge  in  an  audiovisual  presentation 
as  a  result  of  the  insertion  of  learning  objectives. 

2.  There  1s  no  significant  difference  in  cognitive  learning  outcomes  of 
Incidental  knowledge  in  an  audiovisual  presentation  as  a  result  of 
the  insertion  of  learning  objectives, 

3.  Inserting  learning  objectives  at  the  beginning^  at  the  endj  or 
Interspersed  during  an  audiovisual  presentation  has  no  significant 
difference  In  cognitive  learning  outcomes  of  either  Intended  or 
Incidental  knowl edge. 

In  the  analysis s  incidental  and  Intended  learning  were  considered  separately 
using  a  two-way  analysis  of  variance  with  treatment  group  and  sex  as  factors. 


The  subject's  grade  point  average,  class  standing,  and  study  major  were 
included  as  covariates  in  the  analysis.    In  the  event  the  ANOVAs  revealed 
significance  for  either  a  main  effect  or  the  Interactioni  Scheffe's  test  of 
multiple  comparisons  was  used  to  Identify  the  source  of  the  significance.  The 
acceptable  level  of  significance  was  set  at  a  ^  .05  throughout  the  analysis. 

Intended  Learning 

Hypothesis  one:    There  is  no  significant  difference  in  cognitive  learning 
outcomes  of  objective  relevant  (intended)  knowledge  in  an  audiovisual  pre- 
sentation as  a  result  of  the  insertion  of  learning  objectives. 

H      :     0E.  ^  aE    =  0 
0^  J         C  - 

where  Ej  -  experifnental  groups  I,  2^  and  3  and      =  experlrfiental  control. 

The  null  for  research  hypothesis  one  was  rejected  for  both  the  immediate 
and  delayed  post-tests.    Intended  learning  was  significantly  greater  at  the 
.05  level  when  learning  objectives  were  present.    There  was  virtually  no 

variation  in  scores  attributable  to  sex  differences ^  and  there  were  no  signi- 
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ficant  interactions  between  treatment  and  sex.      The  covariate  of  class 
standing  were  also  not  significant.    However^  the  grade  point  average  (GPA)  of 
the  subjects  was  a  significant  covariate  with  the  Intended  learning  score  for 
both  the  immediate  and  the  delayed  post-test. 

Since  no  sex  differences  were  Indicated  in  the  analysis^  sex  was  deleted 
from  all  subsequent  analysis  of  the  intended  learning  variable, 

Incidental  Learning 

Hypothesis  two:    There  is  no  significant  difference  in  cognitive  learning 

outcomes  of  incidental  knowledge  in  an  audiovisual  presentation  as  a  result  of 

the  insertion  of  learning  objectives. 

hi      :  0E.  =  oE    «  0 
Og         J  c 

where  E.  *  experiniental  groups  1,  2,  and  3  and  E    -  experimental  control. 


The  null  for  research  hypothesis  two  was  accepted  for  both  inifnediate  and 
dalayed  tests,  but  the  trend  of  the  results  was  the  reverse  of  the  expected 
direction.    Although  not  statistically  significant  {a  -  ,12  and  a  -  .  10  level 
for  iiiiniediate  and  delayed  scores  respectively)       f^idental  learning  tended  to 
increase  when  learning  objectives  were  prese         nis  trend  was  present 
whether  the  learning  objectives  were  presentt^    efore  the  presentation  (advance 
organizers)  or  after  the  prosentation  (post  organizers),  but  was  not  present 
when  the  learning  objectives  were  interspersed  within  the  presentation  (adjunct 
organizers).    On  the  contrary^  incidental  learning  was  slightly  lower  for  the 
adjunct  organizers.    As  1n  the  intended  learning  outcomes,  sex  differences 
were  not  significant  nor  was  class  standing  significantly  related  to  the 
incidental  learning  scores.    As  with  intended  learning^  grade  point  average 
was  a  significant  covariate  (a  -  .001)  with  the  incidental  learning  scores  and 
appeared  to  account  for  much  of  the  variance  for  both  the  Immediate  and  delayed 
incidental  post-test  observations. 

Location  of  Objectives 
Hypothesis  three:    Inserting  learning  objectives  at  the  beginning^  at  the 
end,  or  interspersed  during  an  audiovisual  presentation  has  no  significant 
difference  in  cognitive  learning  outcomes  of  either  intended  or  incidental 
know! edge. 

Since  the  analysis  of  variance  for  incidental  learning  on  both  the  immedi- 
ate and  delayed  post- tests  showed  no  significance  for  treatment  main  effects^ 
further  testing  between  groups  was  not  warranted*    Consequently,  the  null 
hypothesis  for  the  incidental  knowledge  portion  of  hypothesis  three  was  accepted 
by  defaulti  i.e. s  the  location  of  learning  objectives  has  no  significant 
effect  on  incidental  learning  performance* 


For  the  intended  k row! edge  portion  of  h>pothes1  s  three,  the  slgnifi'cancfi 
of  the  differences  in  pfirforrnance  was  tested  using  t*ie  Scheffe  mathod: 

H       :  uK.    *  pK  =  0 

H.       :  uK.    -  pK   =  0 

where      «  iritended  knowledge  for  experlmertal  groups  1  ,  Z,  and  3i  arid  K_  ^ 

intended  knov/ledge  for  experi  menta  1  control. 

The  riull  hypothesis  w^s  rejected  for  the  advance  organizer  experifnental 

group,    Placing  thelearnliig  objectiv^es  at  the  beginning  &f  the  audiovisual 
presentation  did  signlf Icartl    i  mproye  the  intended  lea rnl  ng  perforfiiance  on 
toth  the  iriinediate  and  delayed  post»test*    The  null  hypGth^ses  for  use  of  the 
learning  objectives  as  a  pest  organizer  and  as  adjunct  org^inl  aers  wre  accepted, 
^Although  tfie  direction  of  intended  learning  perforniance  wa^  greater  fcr  both 
of  these  treatnients  than  for  the  control  group  receivring  nc  learning  cbjec- 
tiveSf  the  i^esults  were  not  s  ign  ifi  cant  at  the  ^05  leveT. 

The  Effects  of  Grade  Point  Average 
Sinca  the  grade  po1  nt  average  had  shown  a  significant  relationship  in 
both  the  Intended  and  Incidentnl  Icarriing  outcomes  an^l/siSj  it  v^as  deci  ded  t^ 
test  for  possibl  e  1  nteraction  effects  between  the  treatnient  and  GPA,  Speci- 
ficany,  the  questicn  arcse  as  to  whether  the  presence  and  location  of  1  earn- 
ing objectives  had  a  dIfferentlaT  effect  on  high  and  lov/ achievement  students 
as  measuTed  b/  their  overal  1  class  standing. 

To  test  for  the  interact<on  b&twe^n  treatment  and  G^A^  a  two-way  analysis 
c^f  variance  was  conducted  v/1th  both  the  Inmediate  and  delayed  Intendid  and 
incidental  learning  scores.    The  A  and  B  subjects  were  gTouped  as  high  GPA 
studentsgi  and  subjects  with  C  ancJ  D  grades  for  the  course  v/ere  designated  as 
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low  SPA  stwidentsi,    It  was  hypothesized  that  high  GPA  students  wul<i  possess 
3uper-1or  organtziing  skills  as  evidenced  by  their  defnonstrated  abll  Ity  to 
riiasterthe  cu^'^i  cu  lurTi  content  of  the  course.    Therefore,  the  Introduction  of 
learning  objecti  va'S  as  a  device  for  organizing  the  intended  learning  material 
€f  tha  audlov^  1SU  al  presentation  would  have  less  beneficial  effGCt,  Students 
lacking  higtil  j  d'eveloped  internallied  organizing  skills,  on  the  other  hand, 
should  fine!  t^e  learning  objectives  more  facnitative  in  learning  the  objec- 
tlvre  relevant  ina  terlal  presented  fn  the  slide-tape  program  but  not  in  learning 
the  Inclderital  kniov^ledge  presented, 

While  Irttui^lwely  reasonabie,  the  data  did  not  support  this  assumption. 
No  significant  interactions  were  present.  The  grade  point  av^irage  varied  as 
scores  vari  d  ?  but  not  dif ferentlan^^  for  treatments. 

Learning  Retention 
In  e^a  ml  fling  the  means  for  the  immediate  and  delayed  post-test  scores, 
differences  w^^e  noted  In  the  eniount  of  learning  loss  between  the  treatment 
groups*    This  was  particularly^  noticeable  for  Incidental  learning  results 
v^hera  the  r^nl<  ordar  of  groups  three  and  four  were  reversed  between  the  im- 
rned  1at€  and  delayeci  post-test.    Ellis  points  out  that  the  retrie\ral  of  inforrna- 
tlon  f  rom  niieiiio^y  is  not  Independent  of  how  it  is  stored.    He  states,       *  *  1f 

Information  is  stored  in  an  organiiGd  and  systematic  fashloni  it  has  a  better 
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chance  to  bm  retrieved. 

to  tes^  tlie  erfects  of  the  tise  and  placement  of  learning  objicttves  on 
the  rate  of  learning  decays  of  the  intended  and  Incidental  knowledge,  an  anal^^sls 
of  difference  scoere  s  was  made*    A  pa f  red  difference  variable  ws  coTiiputed  for 
e^ch  subject  LfSinig  a  repeated  measures  loss  score  of  D  ^       %^  for  each 
krowledge  t>pe*    A  im-m^  analysis  of  variance  was  conducted  v/1th  treatment 
and  se?<as  %\\e  iriide  pendent  variables.    The  results  of  the  analyses  Indicated 


there  were  no  significant  differences  In  learning  loss  between  the  treatment 
groups  or  the  sexes  for  either  Intended  or  fncidental  knowledge.  Although  no 
conclusions  can  be  drawn,  it  is  Interesting  that  the  group  receiving  learning 
objectives  at  the  beglnnirg  of  the  presentation  as  advance  organisers  had  the 
highest  retention  level  for  intended  knowledge  and  the  lowest  retention  rate 
for  incidental  knowledge,  whereas  the  control  group  that  received  no  learning 
objectives  and  the  post  organizer  group  that  received  the  learning  objectives 
at  the  end  of  the  presentation  had  the  better  retention  performance  for  inci- 
dental knowledge. 

It  is  emphasized  there  m$  n£  significance  in  the  analysis  of  differences 
1n  the  loss  of  learning  between  treatment  groups;  therefores  any  conclusions 
from  these  data  would  be  unwarranted  and  misleading. 

Affective  Variables 

Ten  Items  were  included  in  the  imniediate  post^test  that  were  designed  to 
measure  by  means  of  seniantic  differential  scales  the  attitudes  of  the  subjects 
with  regard  to  the  content  material  and  the  rnanner  of  presentation.  The 
variable  names  and  source  of  data  used  in  the  analysis  are  presented  in  Table 
2  (p,  15)  under  "Affective  domain^'  and  '^Evaluation  of  presentation,'*    A  factor 
analysis  procedure  was  used  to  reduce  the  number  of  factors  that  could  be  used 
as  source  variables  for  testing  the  variance  between  treatment  groups. 

As  a  result  of  the  factor  analysiSt  the  ten  variables  were  restructiired 
into  three  gi^oups  that  were  defined  for  further  analysis.    The  variable 
"optimism"  was  retained  as  a  unique  factor.    The  variables  "importance"  and 
"bias"  were  combined  as  a  new  factor  "content  evaluation."   The  remilning 
seven  variables  were  grouped  into  a  single  factor  designated  "presentation 
evaluation."    The  means  for  these  new  generated  factors  by  trfiatment  group  are 
displayed  graphically  1n  Figure  3, 
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The  optlinism  variable  measured  the  subject's  feelfngs  regardlrig  the 
effects  the  new  contnunlcation  technology  ^ould  have  ori  the  quality  of  our 
future  life.    The  scale  ranged  from  ''greatly  enhanced**  (5)  to  "greatly  degraded'' 
(I),    Generallyt  the  treatment  grOLips  appeared  more  pessimistic  than  the 
control  group  with  the  group  receiving  the  learning  objectives  as  adjunct 
organizers  being  the  most  pessimistic*    The  differences  were  not  dramatics 
however,  since  less  than  one  point  on  the  five  point  sc^le  (high  -  3.44,  lov/  * 
2*70)  separated  the  extremes* 

The  range  for  treatment  group  maans  v/as  even  Isiss  cn  the  content  avalu- 
atton  factor  (high  ^  4.41,  low  -  4,07)*  All  of  the  experlfriental  groups  con» 
sldered  the  subject  matter  of  Importance  1n  that  coraiunlcation  inedla  would 
have  an  expanding  role  and  m  Increasirigly  dominant  Impact  on  society.  They 
also  perceived  the  presentation  as  being  relatively  unbiased  in  presenting  a 
balanced  treatnient  of  the  dangers  and  benefits  of  the  new  media  technology  as 
evldencecl  by  the  four  plus  mean  ratings  on  a  five  point  scale. 

The  greatest  range  between  the  experimental  groyps  occurred  in  the  evalu- 
ation of  the  presentation*    Each  subject  rated  the  presentation  on  how  they 
liked  It  overall,  how  appropriate  it  was  for  the  course^  the  content  organlza* 
t1onp  the  length,  the  quality  of  the  visuals  presentedj  whether  they  would 
have  preferred  a  different  format^  and  on  how  interesting  it  was  compared  to 
other  presentations^    Although  the  mean  rating  for  all  four  groups  was  generally 
favorable,  there  was  almost  a  full  point  difference  between  the  highest  mean 
of  4,05  for  the  group  receiving  the  learning  objectives  at  the  beginning  of 
the  presentation  and  the  lowest  mean  of  3,08  for  the  group  receiving  the 
learning  objectives  as  adjunct  organizers  throughout  the  presentation  *  It 
appears  that  Insertliig  the  learning  objectives  during  the  presentation  may 
have  disrupted  the  content  organization  and  created  smB  annoyance  1n  the 
viewers. 
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To  test  the  significance  of  the  difffirences  between  the  experimental 
groups'  affective  and  evaluation  scores,  two-way  analyses  of  yariance  were 
conducted.    The  Independent  variables  were  treatment  and  sex  and  the  dependent 
yarlables  were  the  optimism^  content  evaluation,  and  presentation  evaluation 
variables  generated  hy  the  factor  analysis. 

The  analysis  indicated  there  were  no  significant  differences  in  the 
subjects'  scores  on  optimism  and  content  evaluation  between  the  experimental 
groups.    Female  participants  were  significantly  riore  pessimistic  about  the 
future  degradation  of  the  quality  of  lif^  throuyh  the  improper  exploitation  of 
new  communication  technology  than  ware  the  male  subjects,    Howeyer,  since 
there  was  no  significant  Interaction  with  the  tr^atment^  the  result  was  of 
little  interest  to  this  study* 

There  was  no  significant  variance  1n  content  evaluation  scores  attribut- 
able to  either  treatment  or  sex  differences, 

The  evaluation  of  the  presentation  Itself  did^  however,  show  significant 
treatment  effects.    Sex  had  no  apprecUble  effects  on  the  presentation  evalu- 
ation scores,    A  Scheffe  test  of  the  differences  In  experlrfientai  group  means 
verified  the  difference  between  the  advance  organizer  and  adjunct  organizer 
group  was  significant  at  the  ,05  level,    None  of  the  other  comparisons  was 
significant. 

The  results  of  all  statistical  analyses  are  summarized  In  Table  4, 

Ducussfon 

The  findings  of  this  study  generally  support  and  extend  Ausubel's  theory 
of  advance  organizers  facilitating  learning*    They  are  consistent  with  results 
obtained  by  Kaplan Kaplan  and  Sirnmons,^^  and  ICeyser^^  regarding  the  efficacy 
of  using  learning  objectiyes  as  advance  organizers  with  prose  materials.  The 
length  of  treatment  used  in  this  experiment  was  relatively  short,  but  since 
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Table  4 

Smmry  of  Statistical  Analysis 


Dependent  Variabli 


Immed,       Irmied,       Delayed      Delayed  Analysis 
Int.  Inc.  Int,  Inc.  Method 

Learni ng   Learni ng     Learning     Learni  ng 


Experimental  Factors 


Treatment 

S1g.* 

N.S.^ 

c.  a 
Sig. 

N.S.^ 

ANOVA 

Sex 

N.S. 

N.S. 

N.S. 

N.S. 

ANOVA 

Treatment  x  Sex 

N.S. 

N.S. 

N.S. 

N.S. 

ANOVA 

GPA 

Slg. 

Sig. 

Sig. 

S1g. 

Covariate 

Treatment  x  GPA 

N.S. 

N.S. 

N.S. 

N.S. 

ANOVA 

Col  lege  Major 

N.S. 

N.S. 

N.S. 

N.S. 

Covariate 

Class  Standing 

N.S. 

N.S. 

N.S. 

N.S. 

Covariate 

Retention 

Int.  Learning  Loss      Inc.  Learning  Loss 

Treatment  N.S.  N,S,  ANOVA 

Sex  N.S.  N.S.  ANOVA 


Independent 
Variable 


Comparison  Tests  (Schgffe) 

Dependent  VaHable 

Intended  Learning  ^ 
(Immediate  and  Delayed  Post-test) 


Presentation  Evaluation 


Adv,  Org.  >  Control 
Adv,  Org,  >  Adj.  Org. 

Adv,  Org,  >  Adj.  Org, 


^Test  for  hypothesis  one:  null  rejected, 
^Test  for  hypothesis  two:    null  accepted, 

^Only  those  comparisons  significant  at  the  ,05  level  are  shown* 

^Test  for  hypothesis  three:  null  rejocted  for  intended  liarnlng; 
null  accepted  for  incidental  learning. 
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most  audiovisual  presintatlons  used  as  curriculum  support  are  of  relatively 
short  duration  (one  hour  or  less  in  length),  it  would  appear  that  the  efficacy 
of  learning  objectives  used  as  an  advance  organizer  has  strong  support. 

Locating  the  learning  objectives  at  the  conclusion  of  a  slide-tape  pre- 
sentation or  Interspersed  throughout  with  the  relevant  content  did  not  have  a 
significant  effect  on  improving  either  the  intended  or  Incidental  learning 
outcomes,.    Although  the  mmn  scores  of  the  post  organizer  group  were  higher 
than  those  for  the  group  receiving  no  learning  objectives,  the  lack  of  signifi- 
cance limits  the  support  of  this  study  for  the  use  of  learning  objectives  for 
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mathemager^lc  activities  as  defined  by  Rothkopf/      Previous  attempts  to  extend 
the  mathemagenic  activities  hypothesis  to  audiovisual  media  have  been  ambiguous. 
Dayton  concluded  that  the  use  of  either  Inserted  or  grouped  post  questions 
would  Increase  Intended  learning,  but  that      .  .  the  use  of  either  Inserted 
or  grouped  factual  knowledge  post-questions  In  fixed-paces  fixed-sequence 
preientations  would  not»  by  itself,  ensure  an  Increase  in  incidental  learning, "^^ 

Based  on  the  results  of  the  present  study.  It  appears  the  use  of  learning 
objectives  as  post  organizers  or  adjunct  aids  are  what  Rothkopf  called  "mathe- 
magenic neutral j"  that  Is,  they  have  no  observable  effect  on  the  student's 

A  possible  explanation  of  the  lack  of  facilitation  of  learning  when  the 
learning  objectives  were  used  as  adjunct  organliers  lies  In  the  nature  of  the 
medium.    The  audiovisual  program  used  for  the  stimulus  response  was  well 
organized^  tightly  structured,  with  a  high  degree  of  aesthetic  quality*  The 
Insertion  of  the  learning  objectives  during  the  presentation  could  have  been 
very  Intrusive  for  the  vieweri  interrupting  the  narrative  and  visual  flow  of 
the  production  and  generating  frustration  or  anger.    This  could  have  served  as 
an  inhibiting  condition  for  the  arousal  factors  and  attention  devices  of  the 
highly  professional  production  values  contained  in  the  presentation. 
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This  hypothesis  1s  supported  by  the  results  of  the  presentation  evalu- 
ation scores.    In  general,  the  adjunct  organizer  treatnient  group  liked  the 
presentation  least.    Their  scores  were  the  lowest  on  every  presentation 
evaluation  item,  and  they  were  significantly  lower  than  all  three  experimental 
groups  for  organization.    Overall ,  the  advance  organizer  treatnient  group  liked 
the  presentation  more  than  the  other  experimental  groups.    Their  evaluation 
scores  were  significantly  higher  than  the  other  three  experimental  groups  for 
organization,  quality  of  the  visual and  format.    In  addition,  the  advance 
organizer  treatment  group  rated  the  presentation  significantly  higher  than  did 
the  adjunct  organizer  treatment  group  for  overall  evaluation,  length,  and 
interest. 

The  lack  of  significant  difference  1n  the  Incidental  learning  outcomes  is 

39 

of  some  Importance,    It  contradicted  findings  by  Kaplan  and  Burgln     that  the 

use  of  learning  objectives  decreased  Incidental  learning  with  both  prose  and 

videotaped  Instructional  materials* 

The  effectiveness  of  learning  objectives  to  improve  Intended  learning 

performance  fram  text  materials  has  boen  well  documented  in  thfi  literature. 

The  general  hypothesis  has  been  that  the  increased  learning  stems  from  the  use 

of  the  objectives  by  the  learner  as  directions  to  learn  particular  topics 

within  a  text  (Rothkopf  and  Kaplan*^^  Kaplan  and  Rothkopf,*^  Kaplan/^  and 
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Duchastel  and  Brown    )*    The  concept  that  they  serve  as  orienting  stimuli  that 
permit  selective  attention  to  the  relevant  materials  of  the  text  was  supported 
by  data  showing  students  took  niore  time  when  learning  objectives  were  provided 
(for  review  of  relevant  material)  and  that  performance  on  incidental  learning 
Items  for  students  not  receiving  objectives  was  much  greater.    This  was  ex- 
plained as  being  the  result  of  the  students'  reading  non-selectively,  i.e*^ 
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giving  equal  attention  to  all  material  when  learning  objectives  are  absent* 
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The  results  of  this  study  appear  to  support  a  different  interpretation  of 
how  learning  objectives  are  used  by  subjects  when  the  instructional  material 
is  a  fixed-pace,  fixed-sequence  medlunu 

Looking  at  the  trends  In  the  data  and  not  rnerely  at  the  significant 
levels  of  effectSs  1t  appears  the  use  of  learning  objectives  improved  learning 
performance  on  the  immediate  post'-test  for  both  the  advance  and  post  organizer 
groups.    It  can  be  speculated  that  in  these  treatments ,  the  learning  objectives 
serve  an  organising  function  by  establishing  a  categorical  structure  for 
examining  the  information  in  the  audiovisiial  presentation.    In  the  case  of  the 
learning  objectives  inserted  at  the  beginning  of  the  presentation,  the  informa- 
tion is  considered  and  coded  or  filed  by  the  viewer  under  the  appropriate 
category  as  it  unfolds.    When  the  learning  objectives  are  provided  at  the 
conclusion  of  the  presentation ^  the  viewer  must  conduct  a  mental  review  of  the 
presentation  reorganizing  the  material  according  to  the  new  structure.  In 
either  casej  the  process  of  considering  the  information  and  determining  objec- 
tive relevance  (Intended  knowledge)  or  nor-relevance  (incidental  knowledge) 
improves  recall  for  the  post-test* 

The  significant  relationship  between  the  grade  point  average  of  a  student 
and  the  score  obtained  on  both  intended  ind  incidental  knowledge  was  not 
surprising*    Better  students  should  be  e^cpected  to  perform  better  than  poorer 
students.    It  was  somewhat  surprising  that  no  Interaction  was  found,  however, 

since  ability  to  organize  material  should  be  one  of  the  factors  1n  learning 
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performance.       It  would  seem  reasonable,  therefore,  that  if  learning  objec- 
tives provide  an  organizing  function  thej^  would  be  more  beneficial  to  those 
students  without  highly  developed  internalfzed  organizing  skillSi  1*e,,  the 
poorer  students.    The  lack  of  interaction  Indicates  this  did  not  happen. 
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Conclusions  and  Implications 
The  most  substantive  finding  m%  that  presentation  of  learning  objectives 
as  an  advance  organizer  facilitated  the  Intended  learning  for  a  slide- tape 
audiovisual  presentation.  Of  particular  interest  was  that  these  findings  were 
also  observed  after  a  two-week  delay*  Thus,  the  facllltatlve  effects  of  prior 
presentatton  of  learning  objectives  was  found  for  delayed  as  well  as  Immediate 
recall , 

Although  no  significant  effects  were  found  for  the  facilitation  of  Inci- 
dental learning j  the  acceptance  of  the  null  hypothesis  was  of  considerable 
importance  for  Instructional  strategy.    The  conclusion  drawn  1s  that  Instructors 
may  present  learning  objectives  prior  to  audiovisual  presentations  with  the 
expectation  they  will  enhance  the  learning  of  the  objective  relevant  material 
without  inhibiting  the  amount  of  non^objective  relevant  material  learned. 
From  the  direction  of  the  datai  it  appears  the  tendency  was  for  the  learning 
objectives  to  improve  rather  than  lessen  incidental  learning  outcomes. 

Also  of  Importance  for  Instructors  and  producers  were  the  findings  that 
the  use  of  the  learning  objectives  as  an  advance  organizer  had  no  significance 
for  the  affective  variables  measured  and  they  did  not  detract  from  the  students* 
evaluation  of  this  presentation.    To  the  contrary^  the  trend  of  the  data 
Indicated  the  students  receiving  the  learning  objectives  prior  to  the  presenta- 
tion liked  the  presentation  more  than  when  no  learning  objectives  were  used. 

It  would  be  unreasonable  to  expect  that  the  producers  of  commercial 
audiovisual  educational  materials  would  provide  learning  objectives  as  an 
Integral  part  of  their  presentations*    Such  materials  are  designed  for  the 
widest  possible  market  and  the  specification  of  objectives  might  limit  the 
application  of  the  material  or*  If  not,  be  so  vague  and  encompassing  as  to 
negate  their  effectiveness*    In  addition,  there  is  a  great  deal  of  audiovisual 
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material  (Including  television  programing)  that  can  be  used  for  curriculum 
support  although  It  1s  not  designed  for  educational  purposes. 

Learning  objectiyes  can  be  developed  by  instructors  for  the  audiovisual 
materials  they  use*    the  insertion  of  the  learning  objectives  prior  to  the 
presintation  of  the  audiovisual  prograrn  can  be  accomplished  rather  easily, 
the  primary  concern  of  the  user  of  audiovisual  materials  1s  to  Increase 
Intended  learnlngi  the  insertion  of  learning  objectives  prior  to  the  pre- 
sentation Is  recommended. 
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A  Comparison i 
Predicting  College  Level  Academic  Success 
With  Tests  of  Cognitive  Style  and  Cognitive  Aptitude 

This  study  attempted  to  establish  the  relationship  of  cognitive 
aptitudes  and  styles  to  measures  of  general  and  more  subtle  academic 
achievement.      It  was  expected  that  the  associations  of  aptitudes 
and  styles  to  two  general  measures  of  achievement  would  converge 
to  form  similar  correlational  patterns;   associations  of  aptitudes 
to  more  subtle  measures  of  achievement  would  also  converge,  while 
associations  of  styles  to  subtle  achievement  variables  would 
diverge.     Relationships  with  the  subtle  achievement  measures  were 
as  expected,   but  the  associations  with  general  achievement  measures 
did  not  converge.     It  is  suggested  that  future  style  research  focus 
on  simpler  variables  theoretically  linked  to  mediational  process 
in  the  style  literature* 
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I n  troductiqn 

In  the  systematic  development  of  instruction,   one  major 
concern  of  resGarch  into  individual  differences  has  been  the  attempt 
to  relate  psychological  theory  to  the  prediction  of  college  level 
academic  success  and  ultimately  to  the  effectiveness  of  classroom 
instruction.     Initially  such  prediction  was  used  for  the  purpose 
of  selection;   more  recently,  with  the  development  of  aptitude 
treatment  interaction   (ATI)   research,   aptitude  measurement  has  been 
used  to  optimisse  learning  for  all  students  through  differential 
application  of  instructional  treatments  to  suit  various  individual 
dif f erenceSi 

The  process  of  instructional  design  and  developinent  is  es- 
pecially concerned  with  this  relationship  of  aptitudes  and  treat-- 
ments  when  it  deals  with  the  development  of  academic  courses  in 
higher  education,     A  variety  of  course  designs  such  as  lecture/ 
recitation,   Keller  Plan,   contract  or  tutorial  instruction  may  be 
employed  separately  or  in  combination  as  the  organizational  frame- 
work of  course  format  or  presentation.     A  range  of  learner  aptitudes 
such  as  quantitative,  verbal,   spatial  or  problem  solving  abilities 
may  Interact  with  content  or  course  design  options  to  facilitate 
the  success  of  certain  learners.     For  example,   it. has  been  shown 
that  the  personality  variables  of  achievement  via  . conformance  and 
achiGVement  via  independence  interact  with  certain  course  design 
factors  so  that  students  who  score  high  on  a  measure  of  confomiing 
motivation  demonstrate  higher  achievement  in  settings  which  reward 
conforming  behavior  while  students  who  score  high  on  a  measure  of 


independent  motivation  demonstrate  higher  achievement  in  course 
designs  which  reward  independent  behavior   (Domino,   1968) . 


Problem 


Because  of  the  possibility  of  such  interactions  between 
learner  character  d  sties  and  elemen^'js  of  course  design,   it  is 
important  to  the  field  of  instructional  development  to  investigate 
whether  achievement  in  a  specific  instructional  environment  (es- 
pecially one  with  a  strong  technological  design)   correlates  with 
academic  success  or  whether  certain  learner  variables  are  associated 
with  achievement  in  specific  instructional  designs. 

Traditionally  research  has  examined  the  effects  of  cognitive 
aptitudes  and  personality  dimensions  on  learning.     Recently  interest 
has  grown  in  a  new  category  of  variables,   cognitive  style,  described 
as  an  individual's  characteristic  or  preferred  way  of  perceiving 
and/or  organizing  the  world   (Messick,   1976;   Ausburn  &  Ausburn,  1978). 
In  relation  to  models  of  cognitive  aptitudes  and  personality  dimensions, 
cognitive  styles  are  represented  theoretically  as  mediating  processes 
which  cut  across  the  somewhat  arbitrary  aptitude  and  personality 
categories    (Messick,   1976;   Royce,   3973).     A  number  of  authors  have 
observed  that  this  class  of  variables,   cognitive  styles,   should  be 
useful  In  predicting  academic  success    (Messick,    1976;  Cross,  1976| 
Ausburn  &  Ausburn,   1978).     In  addition,  because  cognitive  styles 
characterize  the  way  in  which  an  individual  learns  rather  than  the 
what  of  learning,   they  should  constitute  an  especially  valuable 
variable  in  the  study  of  learning  In  relation  to  variations  in 
course  designs.     With  theoretical  relevance  to  both  the  processes 
and  the  outcomes  of  academic  performance,  cognitive  style  may  con- 
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tribute  to  the  prediction  of  success  and/or  differential  behavior 
in  a  variety  of  learning  environments. 

The  basic  association  between  cognitive  style  and  the  tradi- 
tional research  variables  of  cognitive  aptitudes  and  personality 
dimensions,   especially  as  they  relate  to  the  prediction  of  academic 
success^   however,   is  largely  uneKplored  and,   therefore^   an  impor- 
tant point  of  departure  for  research-     This  study  approached  the 
fundamental  question:     In  the  prediction  of  academic  achievement, 
both  general   (Grade  Point  Average)   and  at  the  level  of  course 
design,   do  a  selected  class  of  cognitive  style  measures  add  to  the 
variance  accounted  for  by  traditional  cognitive  aptitude  measures? 
This  general  research  question  was  reduced  to  the  following  more 
specific  questions i 

1.  Do  cognitive  style  measures  contribute  tu  the  prediction 
of  general  academic  achievement  (GPA)? 

2*     When  considered  together  do  cognitive  style  measures  pre^ 

diet  general  academic  achievement  over  and  above  traditional 
cognitive  aptitude  measures? 

3.  Do  cognitive  style  measures  contribute  to  the  prediction 
of  academic  achievGment  in  a  specific  course  design? 

4.  When  considered  together  do  cognitive  style  measures  pre= 
diet  academic  achievement  in  a  specific  course  design  over 
and  above  traditional  cognitive  aptitude  measures? 

5.  What  is  the  shared  variance  between  style  and  cognitive 
aptitude  measures  in  predicting  academic  achievement 
moasur es? 

6*  Do  patterns  of  prediction  variables  for  general  academic 
success  differ  from  patterns  of  prediction  variables  for 
academic  achievement  in  a  specific  course  design? 
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Related  Research 

Other  than  with  Wltkin's    (1977)   construct  Field  Depsnd^nce- 
Field  Independence,   there  has  been  a  reasonably  modest  amount  of 
research  examining  the  validity  of  cognitive  style  by  academic 
relationships*     Witkin's  review  of  research  is  quite  eKtensive; 
the  major  generalizations  relevant  to  this  study  arei 

1.  Field  Dependent  students  seem  to  attend  to  and  learn 
more  effectively  from  social  cues. 

2.  Field  Independent  students  are  less  sensitive  to  social 
punishment  and  they  saem  to  learn  more  in  the  absence 

of  external  reward,   under  intrinsic  motivating  contingenci 

3.  Field  Independent  students  seem  to  use  more  mediational 
processes  such  as  the  analysis  and  restructuring  of 
stimuli,   especially  in  unstructured  instructional  environ- 
ments , 

The  following  style  dimensions  were  included  as  variables  in 
the  study    (see  Messick   (1977)   or  Ausburn  &  Ausburn   (1978)   for  a 
more  extensive  review) i 

1,     Field  dependence- field  Independence  is  a  measure  of  an 
individual's  self,   non-self  segregation  as  reflected  by 
reliance  on  internal  versus  external  cues  in  relating  self 
to  the  surrounding  environment*     Behaviorally ^  field 
dependence  influences  the  extent  to  which  an  individual 
operates  autonomously  in  interpersonal  relations  and 
the  manner  in  which  one  processes  information  from  the 
environment.     The  test  used  to  measure  field  dependence- 
field  Independence  was  Witkin's  Group  Embedded  Figures 
Test*     It  involves  disembedding  a  simple  figure  from  a 
complex  pattern p 


2.  Closure  flexibility  measures  an  individual's  ability  to 
keep  a  configuration  in  mind  in  spite  of  distraction. 
There  Is  some  claiin  that  it  is  a  measure  of  temperament 
and  that  ±t  differentiates  industrial  occupation  groups. 
The  test  which  measures  closure  flexibility,  the  Concealed 
Figures  Test,   is  very  similar  to  the  Group  Embedded  Figures 
Test  and  has  been  used  as  a  measure  of  field  dependence- 
field  independence, 

3.  Visualization  and  spatial  relations  were  measured  using 
Flags  and  the  space  relations  scale  of  the  Differential 
Aptitude  Test.     Flags  involves  the  rotation  and  matching 
of  a  two  dimensional  design  while  the  DAT  requires  mental 
folding  of  a  two  dimensional  pattern  to  form  a  three 
dimensional  shape, 

4.  Conceptual  differentiation  is  an  individual's  character- 
istic tendency  to  employ  relatively  more  or  fewer  distinc- 
tions between  concepts  in  category  formation  on  a  free 
categorizing  task.     The  Clayton- Jackson  Object  Sort,  used 
to  measure  conceptual  differentiation  and  compartmentali- 
zation,   is  a  free  categorizing  task  in  which  individuals 
group  fifty  objects  into  an  unspecifieQ  number  of  categories 
and  label  the  logical  reason  for  category  membership. 

5.  Compartmentalization  reflects  an  Individual's  tendency  to 
conceptualize  objects  in  discrete,  rigid  categories.     It  is 
assumed  to  reflect  an  inability  to  produce  diverse  ideas. 
The  measurement  of  compartmentalization  is  a  count  of  the 
number  of  objects  which  an  Individual  is  unable  to  incorpor- 
ate into  categories  in  the  Clayton-Jackson  Object  Sort. 
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Conceptualizing  Style  indicates  an  individual's  character'^ 
istic  manner  of  establishing  criteria  for  category  form» 
ation,   either  on  the  basis  of  physical,  inferred  or  relational 
siinilarity   (such  as  two  dissimilar  objects  each  related 
to  a  third).     The  measure  of  aonceptualizing  style  was  a 
panel  judginent  of  the  category  reasons  on  the  Clayton- 
Jackson  Object  Sort  * 

Leveling-'Sharpening  refers  to  an  indi\ridual' s  mode  of 
organizing  a  sequence  of  stlmali  as  either  relatively 
more  or  less  highly  articulatad.     The  Schematizing  Test 
consists  of  a  series  of  ninety  sguares  varying  in  size 
from  one  to  eighteen  inches.     As  the  squares  are  projected 
on  a  blank  screen  in  a  darkened  room,  the  individual  is 
asked  to  judge  the  size  of  the  squares  in  inches,  A 
"sharpener"  tends  to  make  more  distinctions  between  stimuli 
of  similar  but  unequal  size  while  a  "leveler"  tends  to  group 
similar  sizes  into  a  sirigle  category  reporting  unequal 
squares  as  the  same. 

Category  width  or  equivalence  range  describes  whether  an 
individual  tends  to  form  broad/   inclusive  categories  or 
narrow,   exclusive  categories.     The  instrument  used  to 
measure  category  width  is  the  pettigrew  Category  Based 
Width  Scale.     The  individual  is  asked  to  estimate  the 
largest  and  smallest  example  of  an  item  in  existence  when 
given  an  indication  of  an  average  size.     Broad  categorizers 
will  choose  extremes  while  narrow  categorizers  will  estimate 
closer  to  the  average. 

Complexity-simplicity  rneasures  the  capacity  of  an  individual 
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to  view  the  envirorment ,  especially  the  social  environ-* 
nient  in  a  multldinieiisional  way,     Kelly's  Role  Construct 
Repertory  Test,  modified  for  analysis  via  multidimensional 
scaling  served  as  the  measure  of  complexity^simplicity. 
Participants  were  asked  to  rate  fifteen  paired  roles  as 
similar  or  dissimilar  to  each  other*     CompleK  individuals 
made  finer  distinctions  between  roles ^  using  more  dimen-- 
sions  to  differentiate. 
10,     Constricted  versus  fleKible  control  describes  the  extent  to 
which  an  individual  is  able  to  restrict  his/her  attention 
and  control  susceptibility  to  distraction,   .The  test  which 
Tneasures  this  style  is  the  Speed  of  Color  Discrimination 
Test,     In  the  first  part  of  the  test  participants  are  to 
identify  the  color  of  inK  used  to  print  short  series  of  dots. 
The  test  is  highly  speeded;  participants  are  given  only  fifty 
seconds  to  identify  the  color  of  ninety-six  patterns.  On 
the  second  part^   instead  of  dot  patterns,   the  name  of  a  color 
is  written  in  a  different  color  ink.     The  individual  is 
to  identify  the  color  of  the  ink.     More  flexible  individuals 
are  less  distracted  so  that  the  difference  in  their  scores 
on  the  two  parts  will  be  less. 


Method 

The  college  level  course  from  which  the  participants  for  this 
study  were  drawn  was  a  large-enrollment  introductory  Geography 
course  for  freshitien  and  sophomores  at  a  large  mid-'Western  university. 
The  course  design  was  an  auto-- tutorial  mastery  design  which  has  been 
rigorously  developed  over  a  period  of  five  years. 
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Instructional  environments  consisted  of  a  laboratory  setting 
in  which  students  utilized  behavioral  objectives^   audio  tapes, 
various  visual  stimulus  materials  and  practice  tests  in  a  typical 
mastery  pattern*     A  discussion  section  was  used  for  motivational 
GKerciseB   (e,g,   educational  games)   and  clarification  of  content. 
Previous  evaluation  efforts  have  shown   (Schwen  and  Keller/  1977) 
that  the  treatrnent  is  quite  demanding ^   eliciting  many  productive 
learning  behaviors. 

Course  grades  were  based  on  three  objective  eKams  and  a  judged 
term  paper*     The  exams  were  rigorously  developed  in  the  process 
of  course  development.     Most  items   (twenty  out  of  thirty)  were  of 
a  concept  learning  form  while  the  paper  was  a  modified  problem 
solving  task. 

Participants  in  the  study  consisted  of  240  undergraduate 
students^   two^ thirds  of  whom  were  enrolled  in  the  course  during  the 
process  of  the  study  and  one-third  of  whom  were  a  random  sample  of 
students  who  had  taken  the  course  in  prior  semesters.     These  groups 
of  students  were  compared;   there  were  no  major  differences  between 
the  two  groups.     This  group  of  students  has  been  shown  in  previous 
research  to  be  similar  to  the  general  population  of  college  freshmen 
and  sophomores   (Schwen  and  Keller^   1977) , 

These  students  were  tested  using  conventional  large  group 
testing  procedures.     The  group  of  students  who  had  taken  the  course 
in  past  semesters  were  tested  in  two^   three--hour  evening  sessions 
while  students  enrolled  in  the  course  were  tested  in  a  series  of  nine, 
forty'-minute  testing  sessions  spread  over  a  fifteen  v?e,ek  span.  The 
order  of  teshs  was  varied  to  reduce  the  effects  of  fatigue  and  practice 
as  well  as  any  interactive  effects  between  measures.     Those  subjects 


for  'whont  test  data  was  incomplete  were  dropped  from  the  study. 

Tests  included  measures  of  cognitive  aptitude^  cognitive  style 
and  academic  success.     SAT  scores  and  a  ratio  variable  of  high 
school  rank  to  class  size  were  used  as  traditional  or  "markers" 
in  cognitive  aptitude  predictors  of  academic  achievement. 

Cognitive  style  constructs  \<ere  selected  on  the  basis  of  factor 
analytic  studies   (Royce,   1973;   Gardner,   et,  al.^   1968),     The  criteria 
used  to  select  the  measures  \^ere  that  measures  should  be  reasonably 
well  defined  and  coordinate^   independent  of  one  another;  capable 
of  being  administered  in  large  groups;   reasonably  short;  appropriate 
for  college-aged  participants  and  reliable. 

Measures  of  the  dependent  variable^   academic  success,  were  of 
both  a  general  and  a  course  specific  nature.     The  general  measure 
of  academic  success  was  the  students'   first  semester  grade  point 
average.     Course  specific  measures  of  academic  success  included  the 
second  objective  test  and  a  term  paper.     The  tests  and  processes 
of  judging  papers  were  developed  along  with  the  course  over  a  five 
year  period  and  reliabilities  exceed  traditional  standards  for  both 
measures   (,85  -  thirty  item  testj    ,92  -  inter-rater  reliability  on 
paper) , 

Designi 

This  study  was  the  first  of  several  establishing  the  relation- 
ships between  style  and  instructional  outcomes.     The  dependent 
variables  were  selected  to  represent  the  full  range  of  performance 
from  complex  constructs  such  as  Grade  Point  Average   (GPA)   to  concept 
and  problem  solving  behaviors. 

Traditional  cognitive  predictors  of  achieveinent  were  used  as 
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"markers"  or  bench  marks  for  the  observed  relationships*     This  use 
of  traditional  predictors  made  it  possible  to  compare  the  utility 
and  explanatory  power  of  adding  style  constructs  to  predictors  of 
achieveinent , 

Some  criticisms  of  style  research  have  suggested  these  measures 
are  confounded  with  general  cognitive  ability  measures*     The  use 
of  multiple  stepwise  regression  procedures  permitted  a  partial  test 
of  this  criticism.     The  overlap  between  style  and  cognitive  measures 
was  examined  with  each  prediction  of  a  dependent  measure. 

Our  expectations  regarding  the  data  were  that  the  more  compleK 
composite  variables,   GPA  and  the.  total  points  in  the  course^  would 
illustrate  similar  paLterns  of  relationships  in  combination  with 
the  aptitude  measures.     This  convergent  expectation  or  hypothesis 
was  based  on  the  assumption  that  these  variables  were  roughly  equal 
in  complexity  and  a  result  of  the  saine  complex  processes.     Also,  we 
did  not  expect  style  measures  to  be  highly  correlated  with  these 
dependent  variables.     We  reasoned  that  these  variables  were  quite 
different  in  complexity  and  the  theory  of  the  style  measures 
(Messick, 1977)  would  suggest  that  the  style  measures  are  bi  polar 
mediatlonal  processes  where  the  same  outcomes  may  be  achieved 
through  the  different   (bi  polar)  processes. 

Our  expectations  regarding  the  more  subtle  course  dependent 
variables  and  the  cognitive  styles  and  aptitudes  were  that  the 
cognitive  aptitudes  were  so  complex  that  they  would  converge  in 
association  with  the  more  subtle  dependent  variables.     In  other  words, 
general  academic  aptitude  would  not  differentiate  between  more 
subtle  measures  of  academic  achlGVcment,     However,  we  expected  the 
style  measures  to  diverge,  or  illustrate  different  correlational 
patterns,   in  combination  with  the  dependent  variables.     Our  expectation 


was  that  the  more  subtle  style  constructs  and  related  measures 
could  differentiate  between  the  subtle  measures  of  academic  success. 
In  other  words;  we  assumed  the  different  dependent  variables 
involved  different  mediational  responses* 


Data  Analysis 

The  several  multiple  regression  analyses  are  summarized  in 
Figure  1*     The  amount  of  variance  shared  by  the  cognitive  aptitudes 
and  dependent  measures  is  shown  by  tiie  solid  line  joining  the 
^  shape.     The  amount  of  variance  shared  by  the  style  measures  and 
lapendent  measures  is  shown  by  the.  dotted  line  joining  the  O  shape* 
The  amount  of  variance  the  style  measures  shared  with  the  dependent 
meai3ure  after  the  common  variance  between  aptitude  measures  and 
dependent  measures  was  removed  from  the  regression  equation  iS  shown 
by  the  dash--dot  line  joining  the  A  shape* 

The  total  variance  shared  among  cognitive  aptitude  and  style 
measures  with  the  achievement  measures  is  recorded  at  the  bottom  of 
the  chart. 

The  calculation  of  common  variance  followed  the  SPSS  format 
(Multiple  Regression  Subprogram)  with  adjustment  for  error* 

The  individual  cognitive  style  measures  that  accounted  for  the 
greater  part  of  the  variance  in  each  equation  are  indicated  with  a 
letter  code  immediately  adjacent  to  each  percent-of-^variance  data 
point.     The  problem  solving  variable,  shown  to  the  right  on  the 
graph/   is  a  mGasure  from  another  study  we  conducted.     These  are  problems 
from  the  classical  literature  on  problem  solving.     These  data  were 
incorporated  to  demonstrate  the  contrast  in  relationships  between  the 
more  complex  multivariate  course  related  variables  and  the  more 
precise  univariate  problem  solving  variable* 
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The  most  striking  o£  the  several  relationships  would  seem  to 
be  the  interaction  between  the  complexity  of  the  dependent  variable 
and  the  amount  of  unique  variance  between  style  and  the  dependent 
variables.     In  general  it  would  seem  that  the  more  univariant  vari- 
ables would  be  most  suitable  for  future  research  with  the  style 
variables. 

It  should  be  noted  that  the  dependent  variables  are  not 
independent.     Total  is  a  composite  score  that  includes  Tj  (the 
second  objective  test)  and  the  problem  solving  paper.     The  total 
score  was  utilized  to  compare  it  with  GPA.     As  noted,  the  expectation 
was  that  these  variables  would  exhibit  similar  relationships.  This 
is,  obviously,  not  the  case.     We  have  no  explanation  for  this 
discrepancy.     The  measure  is  either  invalid  or  there  is  a  complex 
interaction  between  the  rest  of  the  variables  in  the  composite 
and  paper  and  the  predictor  variables . 

The  magnitude  of  the  relationships  between  style  and  the 
dependent  variables  is  modest  for  the  course  variables.  Also, 
the  overlap  between  style  and  cognitive  variables  could  be  inter- 
preted as  support  for  the  previously  not#d  criticism  of  the  style 
measures.     Our  interpretation  is  that  the  theoretical  definitions 
of  these  style  constructs  include  mediatlonal  or  process  responses. 
Many  of  the  definitions  are  bi  polar,  not  necessarily  implying 
more  or  less  skill  but  different  mediational  responses.  Therefore, 
the  style  by  achievement  validity  coefficients  may  be  expected 
to  be  modest.     Dependent  variables  closer  to  the  mediational 
processes  or  dependent  variables  that  are  instrumental  in  producing 
the  dependent  variables  may  produce  associations  of  greater  magnitude 

Consistent  with  this  line  of  reasoning,  the  rather  substantial 
associations  between  styles  and  classical  problem  solving  items 


may  be  seen  as  evidence  of  the  confounding  of  cognitive  spatial 
aptitude  (DAT,  Concealed  Figures,  etc.  ?re  sometimes  described  as 
spatial  aptitudes)   and  style.     This  topic  is  being  explored  in  more 
depth  in  a  related  study. 

The  order   (in  the  regression  equations)   of  the  style  and 
aptitude  variables  across  the  dependent  variables  is  of  some  interest. 
The  concept  learning  test  two   (T^)  and  the  paper  seem  to  be  drawing 
on  the  same  cognitive  aptitude  but  different  style  variables.  This 
pattern  is  satisfying  because  the  aptitude  constructs  are  defined  as 
quite  gross  or  complex  whereas  the  style  constructs  are  defined  as 
being  more  subtle.     Also,   the  dependent  variables  should  be  drawing 
on  different  cognitive  processes.     The  assumption  was  that  the 
complex  measures  would  converge  and  the  subtle  measures  would  diverge 
in  association  with  reasonably  subtle  differences  in  dependent 
variables . 

In  summary,   this  study  attempted  to  probe  the  validity  of 
cognitive  style  by  typical  dependent  variables  in  the  context  of 
a  mastery  oriented  course.     Our  expectations  were  that  the  associations 
between  complex  variables  GPA  and  total  points  in  the  course  with 
cognitive  aptitudes  and  styles  would  converge  or  exhibit  similar 
correlational  patterns*     This  was  not  the  case^  leading  us  to 
question  the  validity  of  the  total  measure.     Our  expectations  regarding 
the  more  .univariant  dependent  variables,  concept  learning  and  paper 
performarice  was  that  the  association  would  converge  in  respect  to 
aptitude  ;and  the  dependent  measures.     In  other  words  there  would 
be  similar  patterns  of  correlation  across  these  sets  of  variables. 
Further,  our  expectation  was  that  the  style  by  dependent  measure 
associations  would  diverge.     In  other  words  there  would  be  different 
patterns  of  correlation.     These  expectations  were  more  or  less  confirmed 
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leading  us  to  conclude  the  less  complex  style  measures  may  be  more 
closely  linked  to  the  concept  and  problem  solving  variables  as  the 
theory  in  these  two  areas  would  suggest. 

The  magnitude  and  patterns  of  the  relationships  between  style 
and  the  Gourse  dependent  variables  led  us  to  conclude  there  would 
be  merit  in  continuing  this  kind  of  exploratory  effort.  However^ 
our  new  efforts  will  be  focused  on  simpler  theoretical  variables 
that  are  theoretically  linked  to  the  mediational  processes  described 
in  the  theory  of  cognitive  style.     In  conjunction  with  our 
associates  we  are  currently  eKploring  the  association  between  style 
and  instrumGntal  behaviors  such  as  perceptual  attack  skills^  selectad 
discussion  and  problem  solving  behaviors^ 


Figure  L 

Simimary  Table  of  Multiple  Rigression  Analyiei 
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#=  a. SAT  Verbal,  b.SAT  Math,  c. Class  Rank  -  APTITUDES 

0=  a. Closure  flexibility,  b.Coinpartinentalization,  c.Constricted/PleKiblif 

d, Category  Width,  e. Space  Relations,  f .Visualization 
^  =  Varianci  accounted  for  by  Cognitivi  Style  after  Cognitive  Aptitude 
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ABSTRACT 


"Shim  study  invastigated  the  relationship  of  environinental  and 
personal  factors  of  K^12  media  professionals  in  Alabama  and  the 
reported  performance  by  this  population  of  28  instructional  development 
activitios *     Conclusions  revealed  a  positive  relationship  of  competencies^ 
attitude,  budget^  apathy,  and  non--supervisory  time  with  reported  per-- 
formanee  of  instructional  development  activities.     Also  concluded  was 
a  negative  relationship  of  parent  and  student  volunteers  with  the 
dependant  variable*     Recommendations  include  competency  testing  of  media 
professional  preparatory  pragrsm  graduates ,  research  into  empathy 
training #  and  an  Increase  in  non-supervisory  time  for  media  professionals. 
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INTRODUCriON 

Instructional  developnient  is  a  sy^statnatic  approach  to  the  design, 
production,  and  utilization  of  conipLete  systems  of  instruction  (Associ^ 
ation  for  Educational  Corranunications  and  Technology,  1977 i  172).  Within 
this  realm  of  instructional  development,  si5<  sub'-f unctions  are  delineated. 
These  include  Research'^Theory,  Design,  Prod^action,  Evpaluation^Salection , 
Logistics/  and  Utilization-Dissemination   (AECT,  1977 i  164). 

The  role  of  the  K-12  level  media  professional  as  instructional  devel'- 
oper  has  been  explicated  in  textboolcs  for  potantial  media  professionals 
(Brown,  Norberg,  and  Srygley,  1972 1  1;  Erickson ,  19681  lOj  Prostano  and 
Prostano,  1971^  214-44|  Davies ,  1969 j  41-58),  A  Delphi  Study  by  Jettar 
(1972)  predicted  the  assumption  of  the  role  of  instructional  developers 
as  the  major  trend  in  the  future  for  media  professionals.  Finally,  the 
role  of  the  K--12  media  professional  as  instructional  deveioper  was  for-- 
malizad  in  Media  Programs;  District  and  gchool  (?kinerican  Library  Associ- 
ation/AECT,   1975) , 

Studies  have  been  undertaken  attempting  to  determine  the  relationship 
of  several  variables  analogous  to  instructiQnal  dev^elopment  and  a  vide 
range  of  independent  variables.    These  include  SchulEetenberge   (1970)  i 
curriculum  development  activities  with  undergraduate  major ^  similarity  of 
interests  between  media  specialists  and  teacher/  and  eKtroversion;  Maudus 
(1975) -  curriculum  involvement  with  teaching  structures i  Larsen  (1971) * 
role  assignments  with  time,  budget,  and  principal's  perception;  LGepcr 
(1975)  I  quality  of  media  program  w^ith  both  tlie  attitude  of  principal  and 
media  prof essionali  Daniel   (1974) i  allocation  of  responsibility  to  media 


professional  with  submissiveriess  and  accgmodation  on  fcho  part 

of  the  media  prof assional;  and  Kazt  (1977) i  acceptance  of  the  role  of 

instructional  developer  vith  role^taking  ability. 

The  rola  of  the  media  prDfessional  as  instructional  developer  is  a 
relatively  racent  phenomenon.  the  role  of  instructional  developer 

is  promulgatad  by  national  organizations,  the  recency  of  a  Btandard 
definition  and  deSGription  of  activities  involved  in  instructional  devel- 
opment has  precluded  ir\--depth  studies* 


The  literature  of  library  education  and  educational  technology  is 
replete  with  e^chortations  for  the  media  professional  to  become  involved  in 
instructional  development  at  the  school  level.     A  survey  of  the  literature 
yields  a  series  of  personal  and  environmental  factors  that  would  logically 
be  linked  to  instruGtional  develo^ent.    No  study  was  located/  however, 
which  attempted  to  establish  a  telationship  between  personal  and  environ- 
mental data  and  the  perf omance  by  the  media  professional  at  the  K=12  level 
of  the  instructional  developnient  activities  as  delineated  by  the  AECT 
definition. 
Purposes  of  the  Study 

The  purposes  of  the  study  warei 

1,  to  determine  the  relationship  between  personal  and  environmental 
variables  and  the  amount  of  reported  instructional  development. 

2.  to  determine  the  effect  of  the  variables  on  the  reportpd  perfor^ 
mance  of  inatrLictional  development  activities  within  each  of  the  domains 
delineated  by  the  AEcT  definition. 
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3*     to  identify  the  factors  which  may  lead  to  increased  mtiounts 
of  instructional  development  by  K-12  media  professionals. 
Limit  atigng 

This  study  was  restricted  in  the  follov^ing  areas  i 

1.     the  population  was  restricted  to  media  prof assionals  employed 

at  the  K-12  level  in  the  State  of  AlabOTa* 

2*     the  personal  and  environmental  f actors  were  restricted  to  those 

covered  on  the  instruments* 

METHODOLOGY 

This  research  utilized  questionnaires  as  the  data  collection  instru- 
ments*    This  method  was  chosen  in  order  to  collect  a  wide  range  of  data 
on  a  large  population. 
Independent  Variables 

There  were  38  independent  variables  drawn  from  the  environmental 
and  personal  factors  and  con^inations  of  these  factors. 
Dependent  Vari^^e^ 

The  dependent  variable  was  the  mount  of  reported  instructional 
development  undertaken  by  the  media  professional* 
Subjects 

The  subjects  were  a  sample  of  300  drawn  from  K-12  media  professionals 
in  the  State  of  Alabama  as  identified  by  the  State  Departirient  of  Education 
The  subjects  were  chosen  through  a  random  number  process  and  represented 
a  cross-^section  of  grades  served. 
Ins  t  rumen  tat  ion 

Five  instruments  were  used  including i 
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1 .     Media  Professional  Backnround  Da^ta  Questionnaire,     Th Is  is  an 
lnvestigator"*deslgned  instrinnent  which  gathered  personal  information 
from  the  media  professional.     Variables  measured  by  this  instrument 
included  I 

a*  age 

b  *  seK 

number  of  years  of  classroom  experience 
d*     nuiTtoer  of  years  of  school  library  eKperience 
e*     number  of  years  at  present  position 

f,     possession  of  bachelors  i  masters^  and/or  si^^th-year  degree 
g*     years  since  last  coursework 
h*     number  of  design  courses  taken 
i*     nvanber  of  traditional  library  courses  taken 
j .     number  of  education  courses  taken 
number  of  production  courses  taken 

1.  total  number  of  above  courses 

m*     self-crated  Gompetency  in  design  of  instruction 
n*     self-rated  Gompetency  in  traditional  library  skills 
o*     self^rated  competency  in  education  area 
p.     self^rated  competency  in  production 
total  self-'rated  competency 

2 .  Environinental  Data  Questionnaire.     This  investlgator-^designcd 
instrument  gathered  infomation  relating  to  factors  in  the  working  envi-- 
ronment  which  could  affect  instructional  devalopment  activity.  Variables 
measured  by  this  instrument  includidi 

a.     grade  level  served 
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b,     number  of  hours  assignecl  to  library 

Cp     ainount  of  time  each  day  not  involved  in  the  supervision  of 
students 

d.  ratio  of  library  media  professionals  to  students 

e.  location  of  hV  services 

f*     presence  of  a  production  area 

print  budget 
h*     production  supply  budget 
i.     corranarcial  non-=print  budget 
j  *     total  budget 

k.     number  of  services  provided  by  district  central  office 
1.    number  of  production  equipment  items  available 

number  of  professional  journals  read 

arnount  of  teacher  planning  time/day 

teacher  released  time  during  year  for  course  planning 
p.     number  of  paid  aides 
q.     number  of  student  assistants 
r.     number  of  parent  volunteers 

s.     number  of  innovative  teaching  structures  (team  teaching, 
modular  scheduling,  open^space  classrooms^  etc) 

3.     Empathy  Scale.     I^ie  respondent's  empathy  state  was  quantified 
using  the  Empathy  Scale  developed  by  Robert  Hogan  (1969).     The  long  form 
of  this  instrument  consists  of  31  items  froin  the  California  PsycholQ^ical 
Inventory,  25  from  the  Minnesota  Hultiphasic  Personality  Inventory ,  and 
eight  items  from  experimental  testing  forms,     k  test^retest  reliability 
coefficient  of  .84  has  been  found,  and  intDrnal  consistency  estimates  of 
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*71  have  been  reported  (Grief  anil  Hogan,  1973),  The  conceptual  validity 
of  the  scale  has  been  demonstrated  in  several  studies  (Hogan  and  Mankin, 
19701  Hogan,  1973,  Kurtines  and  Hogan ^  1972;  Hogan  and  Dicks tein,  1972). 

The  short  form  of  the  mipathy  Scaley  which  consists  of  only  those 
31  items  from  the  California  Psychological  inventory,  has  routinely 
correlated  above  .90  with  the  64  item  version  (Grief  and  Hogan,  1973 t 
281),     The  short  form  was  used  for  this  study.     The  author  felt  that  the 
use  of  the  siinpler  form  would  significantly  increase  return  rate. 

4.    Media  Professional  Activities  Checklist.     This  instrument  was 
developed  by  the  investigator  and  contained  28  aGtivities  drawn  from  the 
Task  Analysig  Survey  Instri;unent  {hth,  1969)^  from  other  literature,  and 
from  correspondence  with  professionals  in  the  field.     The  respondent 
checked  a  Likert-type  scale  ranging  from  0  to  4  for  each  of  the  28  activ- 
ities*    Anchors  for  these  were  "never"  and  "frequently."    Total  possible 
scores  ranged  from  0  to  112.     Using  the  same  instrument,  the  media  profes- 
sional was  also  requested  to  check  a  Likert-type  scale  to  indicate  the 
perceived  importance  of  each  task.     Anchors  for  this  scale  were  ''little 
or  no  ijnportance"  and  *'very  important," 

The  activities  were  selected  to  provide  a  continuum  of  activities 
within  instructional  development*     In  Educational  Technology i  Definition 
and  Glossary  of  Terms ,  instructional  development  is  divided  into  six 
functions  including  Research^Theory ,  Design,  Production,  Evaluation- 
^Selection,  Logistics^  and  Utilization-Dissemination  (AECT,  1977^  164). 
Each  of  the  28  activities  is  subsunied  by  at  least  one  of  the  instructional 
development  functions,  with  all  six  functions  being  represented  by  at 
least  one  activity. 
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*if\Q  instrumGnt  wm  Boi\t   to    scvGral  madia  profess iona Is  for  tholr 
commenti  utilised  in   a  pl^Qt   otu^dy  (Turner  and  Martin,  1978),  and  rGvisod 
on  the  basis  pf  responses         cla^ta  recGivecL     A  ralinbility  coeftMciant 
of  vao  G^jtablished  uQLxxq    the  Bplit^half  metlxocl. 

Construct  validi.ty'  v/as  anve  otigated   thraugh  the  use  of  the  factor 
analysis  of  the  2b  actl^ities^     Six  factors  were  formefl/  four  of  which 
accoanted  £ox  91*3  percent  o£  th  e  variance  and  included  25  of  the  28 
activities*     A.  reyiev^       the   activities  included  within  each  factor  por-- 
mittcd  the  ass  igjiment  0.5  f^ctoi:    construe  Us  oC  Iuul  of  th^  aix  functions 
of  instmctional  devalapmejit    del  ineated  in  the  OCT  definition.  These 
factor  constructs  inclu^dad  gi^alu  ation^Selection,  Design  ,  Production/  and 
Logistics,    ^he  groupincg  o^  sctin^ities  within  the  jjgd i a_ j r o j^as s iona  1 
Ac tij^i t ies  Che ck list  tixro^Q^j  tea  large  extent,  the  theoretical  group- 
ings proposed  in  the  ASCT  ^ggini  tion  of  instruct ioral  development. 

5-     Pring i pal^* s^jjcce^t^on^Che c k  1  i s t .     This  instruiTient  consisted  of 
the  2  8  activities  foand  on  tt^e  Media  Proffessional  Activities  Checklist. 
The  respondent  indica^tedr  ^asi-ng    aLikert— typa  scale^  the  importance  of 
each  task  being  pertornied        the  media  profess  iona  1  in  the  school.     Pos*-  ■ 
sible  scores  ranged  f  mrn  0  to  ll  2. 
Procedu  jes 

A  packet  contain  i^ig  tli@  ins  truments  and  two  cover  letters  was  sent 
to  the  principal  of  eac^  school  in  the  sainple.  The  initial  cover  letter 
v/as  frorri  the  State  Depa.itiTi^nC  of  Education  eKplaining  the  importance  of 
the  study"  and  requesting  cc^opeta tion.  The  second  letter/ which  was  from 
the  researGherSi  intr odijce^  provided  the  rationale  for  the  study  as 
well  as  supplying  directioTig    for  the  principal.     The  principal  was 
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requGstod  to  forward  the  instrurncints  to  be  complGtQcl  by  the  medio  pro-^ 
fessiorial  and  to  see  that  all  materials  were  rGturnGd*     The  media  profGs^ 
sional'  fj  materials  were  in  a  self-contained  packet  which  was  soalGd  bc^ 
fore  rD turning  to  the  principal  for  mailing. 
Dat-a  Collection 

Ttie  responses  from  the  instruments  were  tabulated  with  most  of  the 
responses  being  used  in  raw  data  form*     A  small  percentage  were  translated 
into  frequency  counts*     A  total  score  was  generated  for  the        a t hy_  S c a  1  o  , 
the  Pr i  nc g  Pe^cejjt ion  Cha ukl  itiLf  ^nd  for  tjauli  uf  the  two  parts  of 
the  Media  Professional  Activities  Checklist,     The  other  instruments  were 
scored  for  eaoh  subsection. 
Analysis  of  the  Data 

The  variables  for  which  dlchotomous  data  were  generated  were  analyged 
by  the  use  of  chi^-square  tests  of  independence.     To  do  this  ^  the  total 
score  f roffl  the  performance  section  of  the  Media  Professional  Actlvl^t^e_s 
Checklist  was  dichotomized  arQund  the  mean  into  0^66  and  67-112  categories. 
For  the  remaining  independent  variables i  a  simple  correlation  coefficient 
was  geaarated  against  the  independent  variable. 

The  28  activities  of  the  Media  Professional  Activities  Chscklis^t 
were  submitted  to  a  factor  analysis  using  the  Varimax  method  of  rotation- 
FactDr  scores  Were  generated  for  each  subject.     Finally^  the  independent 
variaiLes  werQ  run  against  the  factor  score  variables  in  a  stepwise  multi- 
ple regression. 


34: 


RESULTS  A!ID  ANALYSES 

IVo  hundrGd  and  three  questionnaires  were  returnGd.     Of  theaG^  187 
were  complete  enough  to  be  usable, 
pichotomous  Data  Analyg^as 

Seven  of  the  40  variables  yielded  dLchotomouB  data.     These  included 
grade  levels  of  students  served^  student/niedia  professional  ratio,  location 
of  AV  services  in  or  out  of  the  department,  availability  of  a  production 
area,  released  time  for  teachers  to  engage  in  eKtended  planning,  sex  of, 
and  degree (s)  held  by,  media  professionals.     Of  th^se  variables,  Qnly  the 
released  time  for  teachers  for  eKtended  planning  yielded  a  significant 
difference  at  the  .05  level.      (See  Table  1  for  chi^square  values.)  Two 
of  the  remaining  three  variables  were  slg^nificant  beyond  the  .1  level* 
These  were  the  student/media  professional  ratio  and  availability  of  a  pro*- 
duction  area. 

TABLE  1 

RESULTS  OF  CHI-SQUARE  TESTS  OP  INDEPENDENCE 


Variable  ^2  df 


Grade  levels  served  4,715  7 

Student/media  professional  ratio  15,20^  9 

Location  of  AV  services  0,38  2 

Presence  of  AV  production  area  2,70*  1 

Released  time  for  teachers  for  6,16'^^  I 
extended  planning 

Sex  of  media  professionals  1.68  1 

Type  of  degree   (Bachelors ^  Masters,  1,02  5 
or  Sixth^year) 


p  <  .1  p  4.02 
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As  can  be  seen  in  Table  2,  »=he  significant  difference  goneratGcI  by 
the  teacher  released  time  variable  resulted  in  a  greater  number  of  in^ 
structional  development  activities  reported  by  media  professionals  in 
schools  where  teachers  were  given  time  off  during  the  year  for  planning » 
The  presence  or  absance  of  a  production  area^  as  evidenced  in  Table  2, 
also  resultiad  in  a  significant  difference^  although  at  the  .1  level  of 
significance/  in  the  amount  of  instructional  development  reported.  A 
greater  proportion  of  the  media  professionals  who  had  a  separate  produc- 
tion area  reported  that  they  pcrforxned  instructional  dsveloprnent  than 
those  without  suc-h  facilities. 

Results  and  Analyses  of  the  Reinaining  Variables 

A.  Pearson's  r  was  calculated  for  each  of  the  remaining  variables  to 
establish  the  relationship  between  these  variables  and  the  amount  of  in-- 
structional  development  reported.     As  can  be  seen  in  Table  3/  four  of  the 
variables  I  hours  assigned  to  the  media  center,  number  of  student  helpers, 
number  of  parent  volunteers ^  and  age  of  the  media  prof essional,  ware  cor^ 
related  negatively  with  the  amount  of  instructional  development  activities 
reported.     The  correlations^  however ^  are  all  low^  with  only  the  number 
of  student  helpers  generating  a  significant  relationship  and  that  at  the 
»1  level. 

Of  the  29  variables  with  positive  relationships,  21  generated  re- 
lationships significant  at  the  .1  level  or  above.     Variables  significant 
at  the  .01  level  or  above  included  number  of  production  equipment  items, 
number  of  professional  journals  read^  number  of  innovative  teaching 
structures,  all  five  of  the  competency  variables^  empathy ^  the  pr Lnclpal ' s 
attitude  toward  the  performance  of  instructional  dQvelopment  by  the  media 
professional,  ana  the  media  professional's  attitude  toward  instructional 
developinent , 


TABLE  2 


CONTINGENCY  TABLES  FOR  TEACHER  RELEASED  TIME 
AMD  AV  PRODUCTION  AREA  VARIABLES 


Teacher  released 
time 


a  6.18 
p  I  -02 


Instructional  Development 
0-66  67-112 


Yes 

6 

17 

23 

No 

69 

54 

123 

df  1 

75 

71 

14G 

AV  Production 
Area 


Instructional  Development 


0-65 

67-112 

Yes 

28 

41 

69 

No 

49 

40 

89 

77 

81 

158 

df  1 

=  2.71 
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MEANS  AND  STANDARD  DEVIATIONS  OF  CONTINUOUS  DATA 
INDEPENDENT  VARIABLES  AND  r  VALUES  WITH 
REPORTED  INSTRUCTIONAL  DEVELOPMENT 


Variable  Mean  SD  r  with  reported 

instructional 
development 


Kours  assigned  in  media 

center  6.65  1.32  -.0338 

Time  not  assigned  to 

supervisory  tasks  1*40  1.74  .0437 


Enrollment  691.35  410,28  .0403 

Budget  for  print  ^ 

materials  1983,40  1660.41  .1081 

Budget  for  production 

supplies  249.24  411.00  ,1984 
Budget  for  non-sprint 

commercial  materials  612.44  858.00  .1901 

Total  budget  2356.16  2456.66  .1769 
Number  of  services  pro^ 

vided  by  district  1.48  1.35  .0543 

Number  of  production  **** 

equipment  items  1.93  1.45  .3656 

Number  of  professional 

journals  read  3.16  1.63  .2204 


?jnount  of  teachers '  daily 

planning  time   (hours)  .83  .42 


**£  < .05 
***g^  <  .01 
****p  ^.001 


Tabla  3  continued 


Variable 


Mean 


SD 


£  with  reported 

instructional 

development 


Amount  of  paid  help 

(hours  daily)  1.09 

NumbGr  of  atudsnt 

helpers/hour  1.87 
Number  of  parent 

volunteers/hour  ,05 

Number  of  taaching 

structures  1.89 

Age  of  media  professiDnal  40.03 

Years  of  classroom 

eKperience  6.22 
Vears  of  library 

experienca  8,53 

Years  at  present  position  6.55 

Years  since  coursework  3.71 

Number  of  design  courses  5.45 

Number  of  traditional 

courses   (library)  4. 25 

Number  of  education 

courses  3,02 

Number  of  production 

courses  .52 

Total  number  of  courses  13.28 

Competency  in  design  22.74 

Competency  in  traditional 

library  skills  17,42 


1.64 

2.17 

.26 

1.39 
11.47 

7.86 

6.22 
5.37 
3.94 
3.06 

1.20 

1,73 

.76 

5,75 
7.03 

2.87 


.1133 

1084^ 

-,0824 

.1905^ 
-.0960 

.0155 

.  068O 
. 1029* 
.0333 
. 1173* 

.0895 

* 

.  1620 

i 

,  1797 
.1517' 
,  403l' 

. 3218' 


**£  <',05 


***p  <.01 

****£  <.0Ol 


Table  3  continued 


Variable  Mean  SD  r  with  reportQcl 

instructional 
development 


**** 

Competency  in  education  11.19  4.10  .3674 

*  ^  * 

Competency  in  production  2.64  2,05  .  2285 


Total  competency  53.64  12.83  .4483 

Elnpathy  15.87  3.69  .2224' 


*  *  * 


Principal's  attitude  tov/ard 
media  prDfessional  perfor^ 
mance  of  instructional  deve^ 
lopment  89.02  13.50  .3559 


Media  professional's  atti^ 
tude  toward  the  perforTnance 
of  instructional  develops 

ment  90.18  17.27  .4084 


<.05 

**  * 

'***£  <  ,001 
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A  factor  analysis  of  the  Media  Professional  A^ti^^^  Chedklist 
utilising  th^  V^aritnaK  Method  was  undertaken.     This  analysis  resulted 
in  the  formatiDh  of  four  factors  including  25  of  the  28  activities  and 
accounting  for  91.3  percent  of  the  variance   (See  Table  4), 

TABLE  4 

FACTOR  GROUPINGS  OF  INSTRUCTIONAL 
DEVELOPMENT  ACTIVITIES 


Factor  1  Evaluation-^Selection 


Verbally  suggests  titles  of  instructional  materials  to  teachers 

Determines  ttieasurable  objectives  for  media  program 

SQlicits  feedback  from  teacher-s  via  forms  and  interviews  about 
materials  produced  and  purchased 

Validates  the  instructional  materials  purchase-d  and  produced  by  using 
observatioris  of  student  behavior  and  matching  these  with  instructional 
objectives 

Prepares  lists  of  commercial  materials  in  response  to  instructional 
objactivesy   learner  character istics ,  and  other  parameters 

Engages  in  research  activities  relative  to  the  media  center  prograin 

Develops  lists  of  titles  of  instructional  materials  in  response  to  a 
specific  topic  given  by  a  teacher 

Works  with  teachers  in  selecting  materials  to  meet  specific  objectivas 
Eigen  Value    8,6955  Percentage  of  Variance  68,6 


Factor  Design 


Works  with  teachers  in  formulating  instructional  objectives 
Plans  and  discusses  media  center-relatad  topics  with  teachers 
Participates  in  team-teaching  activitias 


TABLE  4  continued 

Observes  classroom  work  to  coordinate  rnedia  center  activities  with 
instructional  programs 

Participates  in  curriculum  development  and  revision 

Plans  jointly  with  faculty  tnentoers  to  coordinate  media  center  activities 
with  curi^iculum  programs^  units ^  and  textbooks 

Condunf.R  workshops  for  teschcrs  in  e'valuatiOii  ^uu  ^eleution  of  materials 
Acts  as  part  of  an  instructional  team  in  designing  instructional  prograins 
Develops  long-^range  plans  cooperatively  with  teachers  and  administration 
Assists  individual  teachers  in  curriculum  planning 

Conducts  workshops  for  teachexs  in  use  of  materials  and/or  equipnient 
Eigen  Value  1.3094  Percentage  of  Variance  10.3 


Factor  3  PrDduction 


Designs  and  produces  instructional  materials  in  response  to  a  general 
topic  given  by  a  teacher 

Designs  and  produces  instructional  materials  in  response  to  instructional 
objectives,  learner  characteristics^  and  other  parameters 

Produces  instructional  materials  in  response  to  a  specific  design  given 
by  the  teacher.     This  design  specifies  the  type  of  medium  and  content  to 
be  included 

Eigen  Value  *8954  Percentage  of  Variance  7.1 


Factor  4  Logistics 


Arranges  and  conducts  sessions  to  preview  and  evaluate  materials 

Gives  instructions  to  students  in  use  of  materials  and/or  equipment 

Maintains  a  source  of  curriculum  guides  and  professional  materials  for 
use  in  planning 

Eigen  Value  ,6717  Percentage  of  Variance  5*3 


Constructs  taken  from  the  sub-functions  of  instructional  doVGlopmunt 
as  dafined  in  Educational  TechnQlogyg  Definition  and  Glosgary  of  Terms 
(aECT,   1977i   164)  were  assigned  to  these  four  factors*     These  included 
Evaluation-Selection,  Design^  Production^  and  Logistics.  Factor^scores 
based  on  the  four  factors  were  generated  for  each  subject, 
Mul tiple  Reg  res 5  ion 

Four  stepwise  regressions  were  run  with  each  factor  utilized  as  a 
dependent  variable  associated  with  the  independent  variables.     These  runs 
were  set  to  liinih  the  indepenuenu  variables  utilised  to  Uh^.j^e  tnat  contrr^ 
touted  at  least  two  percent  additional  variance. 

The  first  f actor  was  assigned  the  construct  Evaluation-Selection, 
The  independent  variables  utilized  in  the  equation #  in  order  entered ,  were 
total  competency^  hours  assigned,  number  of  design  courses,  principal's 
attitude,  number  of  education  courses^  classroom  experience ^  amount  of 
time  not  assigned  supervisory  tasks,  and  competency  in  traditional  library 
skills   (see  Table  5).     Two  of  the  variables,  number  of  design  courses  and 
ajiiDunt  of  classroom  eKperience,  related  negatively  to  the  Factor  1  vari- 
^le.     The  eight  variables  utilized  accaunted  for  54  percent  of  the  vari- 
ance* 

The  second  factor  was  assigned  the  construct  of  Design.     The  indepen- 
dent variables  utilized  in  the  multiple  regression  equation  were  total 
competency,  principal -s  attitude,  total  budget,  oinount  of  time  not  assignGd 
supervisory  tasks,  competency  in  design  areaj.  number  of  education  courses, 
attitude  of  inedia  professional,  and  cqmpetency  in  education  courses  (see 
Table  S) .     Ml  of  the  relationships  between  the  independent  variables 
utillEed  and  the  factor^score  were  positive.     The  resulting  equation 
accounted  for  68  percent  of  the  variance. 
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TAHLE  5 


MULTIPLE  REGRESSION  SUMMARY  TABLE 
FOR  FACTOR  1  (EVALUATION-SELECTION) 


Independent  Variables          Multiple  R      R  Square      R5^  Change      Simple  R 


Total  competency 

,5176 

.2697 

.2697 

Nurnfasr  of  lioucs  flssig^nsd 

,  5565 

3  209 

*  0530 

.  1792 

*  D  i  3  ** 

1  R 1 

*  J  w  J  Q 

*  0S27 

^ , 0268 

Principal's  attitude 

,6627 

.4392 

.0555 

.2972 

Nmnber  of  education  coursas 

.6861 

.4707 

,0314 

.1352 

Classroom  experience 

.7033 

,4947 

.0239 

-.0632 

Arnount  of  time  not  assigned 
supervisory  duties 

.7183 

,5160 

.  0213 

-  0710 

Competency  in  traditional 
library  skills 

.7387 

.  5456 

,  0295 

,4456 

TABLE  6 

MULTIPLE  REGRESSION  TABLE 
FOR  FACTOR  2  (DESIGN) 

independent  Variables             Multiple  R 

R  Square 

RSQ  Change 

Simple  R 

Total  CQmpetency 

.6274 

.3936 

•  3936 

«6274 

Principal's  attitude 

.6884 

,4739 

.0803 

,4190 

Total  budget 

.7222 

,5216 

.0476 

.3024 

Amount  of  time  not  assigned 
supervisory  duties 

-7482 

.5599 

*  0383 

.  1664 

Cornpetency  in  desi^ 

.1151 

.6081 

.0419 

.  5290 

Number  of  education  courses 

,7956 

,6329 

,0311 

,0937 

Media  professional's  attitude 

.8142 

.6629 

.0299 

.1959 

Cornpetency  in  eduaation  area 

,8276 

,6849 

,0220 

.  5432 
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The  construct  of  Production  was  assigned  to  the  third  factor.  Six 
independent  variables  were  utilized  in  the  multiple  regression  equation 
to  eKplain  46  percent  of  the  variance.     The  variables  included  total 
competency,  number  of  parent  volunteerSf  enrolliTient ^  amount  of  time  not 
assigned  supervisory  tasks  /  competency  in  traditioneil  library  skills^  and 
competancy  in  production   (see  Table  7)  .     Tyo  of  the  variables,  number  of 
parent  VDlunteers  and  enrollment/  were  associated  negatively  with  the 
f actor-scora , 

TABLE  7 

MULTIPLE  REGRESSION  SUMmBY  TABLE 
FOR  FACTOR  3  (PEODUCTION) 


Independent  Variable  Multiple  R      t  Square  Change      Sijnple  H 


Competancy  total 

.4852 

.2354 

,2354 

.4852 

Parent  volunteers 

.5930 

,3516 

.1162 

--.3052 

Enrollment 

.6274 

.3937 

,0420 

-,0134 

Amount  of  time  not  assigned 
supervisory  tasks 

.6447 

,4157 

.0219 

.1709 

Competency  in  traditional 
library  skills 

.6512 

.4372 

,0214 

.3046 

Competency  in  production 

.6791 

.4655 

.0283 

.3287 

The  fourth  and  final  factor  was 

ass  igned  the 

construct  of 

Logistics 

Ten  independent  variablas  were  entered  into  the  equation 'resulting  in  the 
eKplanation  of  62  percent  of  the  vaiianca.     These  included  competency  in 
the  education  aroa^  number  of  production  equipment  items  ^  years  of  cliiss- 
room  eKperience^  production  budget^  amount  of  student  help,  amount  of  paid 
he^lp,  number  of  traditional  library  courses,  hours  assigned  to  the  media 


center,  age,  and  years  since  last  course  work.     Four  of  the  variables 
were  related  negatively  to  the  factor  variable.     These  included  years  of 
classroom  experience/  amQunt  of  student  help,  age,  and  years  since  last 
coursework. 


TABLE  8 

MULTIPLE  REGRESSION  SUMMAEY  TABLE 
FUH  ^^ACTOR  4  (LOGISTICS) 


Independent  Variable  Multiple  H      R.  Square      RSQ  Change      Simple  R 


Competency  in  education  .4385  .192  3  .1923  .4385 
Number  of  production 

equipment  items  .5734  ,32S8  .1365  .3781 
Years  of  classroom 

eHperienee  .6248  .3904  .0615  -.1504 

Production  budget  .6528  ,433  2  .0428  .3343 

Student  help  .6822  .4653  .0321  ^.2449 
Number  of  traditional 

courses  ,7317  .5379  .0254  .1147 
Number  of  hours  assigned 

to  media  center  .7474  .5615  .0236  .2773 

Age  .7596  .5858  .0243  -.0571 

Years  since  last  coursawDrk      .7872  .6237  j379  --.0424 


CONCLUSIONS 

The  leap  from  relationship  to  causality  is  a  critical  one  and  can 
only  be  justified  when  the  methodDlogy  is  valid  and  the  instrumentation 
is  .accurate.     Because  this  study  enters  t erra__i n cog n i t a  ^  the  leap  requires 
a  large  assist  from  faith  as  well  as  from  logic.     Some  distinct  patterns # 
however*  have  evolved  out  of  this  study,  and  these  warrant  elaboration. 
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The  varisibles  that  st^nSouifrom  the  others  in  consistency  and  in 
strength  of  gsositive  relatiorsshi^  to  the  amount  cf  reported  instructional 
dsvelopxnent  are  four   cOCTpetermcy  -components  taken  singly?    design,  tradi^ 
tional  ILbrary  s1<ills  /  adulation  /  production,   as  well  as  the  sum  total  of 
thase  components*    An  tjTune^iats  tendency  exists  to  be  sKeptical  about  a 
high  degree  of  coirrel  atdon  b^twe*en  two  measurements  when  the  raw  data 
were  self -reported*     "ihrnte        evidence,  howev^er,   that  indicates  that  use 
of  the  reporting  fom  fcr  Qornpet«§ncies  is  not  necessarily  subject  to  a 
distortion  caused  by  an  iiiQr3ina^ely  favorable  reporting  of  competency  and 
perf ormarLce.     In  a  pilot  s^u3y  (burner  and  Martiiii  1978),  utilizing  a 
populati&n  w£th  rslati^ely  h(^ffog«neous  coinpetencles,  the  competency  vari- 
ablss  did  not  prove  t<  Be  sig^nlf dcantly  related  to  the  amount  of  reported 
instruetLonaL  devtlopinerit . 

•The  first  coricluslon^  thietedore,   is  that  the  reported  competencies  of 
the  media,  prof  eg  slona  1  In  the  ar«as  of  design,  traditional  library  skills, 
educatioai  and  producticn  hav^e  tEne  strongest  relationship  with  reported 
instructional  devslopEnSrit.    rtia   conclusion  is  ^  of  course,  not  surprising 
as  one's  cOrnpetency  i^  an  mx&s^t  eipecially  in  a  predominantly  elective 
sector  such  as  instructional   development,    should  certainly  have  a  strong 
bearing  on  whether  activitigs  ^re  undertaken  or  not.     It  would  be  difficult 
to  envision  a  strong  in-strt4Qt=lorial  developrnent  component  within  a  ^ 
media 'program  if  y^?  Mdia  p^-^ofeseional  lacked  competency  in  any  of  the 
" abo V  e  EoM r  are  as , ^ 

^The  reading  of  prcfession-fli  journals ,    certainly  a  method  of  increasing 
competence*  is  also  pes iti^eLy  related  to  instructional  development*  This 
variable  was  also  significant  in  the  pilot  study  where  continuing  indepen- 
dent study  was  the  main  metho  d  orf  differentiation  of  competency  for  the 
population  utiligefl, 
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The  second  CDnclusion  is  that  a  positive  attitude  by  both  the 
principal  and  the  media  professional  toward  the  inclusion  of  instructional 
developxneiit  in  the  role  of  the  rnedia  professional  increases  the  amount  of 
instructional  dievelopment  undertaken.     Once  again,  this  is  not  a  startling 
concluslDii^  being  consistent  with  the  findings  of  several  previous  studies 
(Larsen /  1971;  Leeper,  1975).     Since  the  performance  of  instructional 
developifleiit  is  so  often  elective,  done  beyond  the  normal  routine,  and 
often  re^jires  a  change  in  adininistrative  factors,  a  positive  attitude  by 
the  media  professional  as  well  as  the  principal  would  obviously  be  bene- 
f icial*  ^ 

The  third  conclusion  is  basad  on  the  low  but  signif leant  relationships 
of  all  four  budget  variables /  as  well  as  the  moderate  relationship  evidence 
with  the  nuinbetf  of  production  equipment  items  prenent.     This  conclusion  is 
that  the  perfomanca  of  instructional  development  requires  a  miniinum  fi-- 
nancial  support  base.     Since  instructional  development  usually  I'equires 
alternative  materials  and  delivery  systems/  as  well  as  the  production  of 
materials  ,  this  rninimum  base  requirement  is  to  be  eKpected. 

The  fourth  conclusion  concerns  the  einpathy  variable,    W^iile  the  re- 
lationsliip  between  the  empathy  score  and  reported  instructional  develops 
ment  is  IcWf  ii  is  significant  and  provides  the  basis  for  the  conclusion 
that  empathy  is  a  positive  attribute  of  the  personality  of  the  instruc- 
tional developer  at  the  K-X2  level. 


The  availability  of  released  time  for  teachers  to  engage  in  extended 
plannin^r  a  variable  which  could  also  be  influenced  by  the  principal's 
attitude,  resulted  in  a  significant  difference  in  the  chi-square  analysis 
and  lends  weight  to  this  conclusion. 


The  fifth  conclusion  is  actually  a  corollary  of  the  second  in  that 
the  principal  usually  makes  the  general  schedule,  and  the  media  profeS'- 
stonal  generates  the  specific  details,     This  conclusion  is  that  in  order 
to  perform  instructional  development  activities^  the  meraia  prDfessional 
must  be  assigned  to  the  center  v?ith  sufficient  time  to  perform  logistical 
support  activities*     More  importantly,  the  media  professional  must  toe 
allowed  time  where  he/she  is  not  involved  in  direct  supervisory  activities 
in  order  to  perform  evaluation,  selectioni  design^  and  production  activ-ities. 

The  sixths  and  final,  conclusion  is  based  on  the  negative  relationship 
found  in  this  study  and  in  the  pilot  study  of  the  pa^Gnt  volunteer  and 
student  help  variables  to  the  reported  arnQunt  of  instcuctional  development. 
'This  conclusion  is  that  volunteer  assistance  in  the  form  of  either  students 
or  parents  does  not  appear  to  contribute  tc  the  amount  of  instructional 
development  undertaken*     This  rathar  surprising  conclusion  may  be  eKplained 
by  the  amount  of  time  that  is  required  to  organize  and  administer  volunteers. 
Rather  than  freeing  the  media  professional  to  perforrn  instructional  deve- 
lopment activities,  energies  appear  to  be  channeled  ihto  supervisory 
activities. 

RECOMMENDATIONS 

Based  upon  the  conclusions  of  this  study,  the  following  recornmendations 
are  offered^ 
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1,  The  graduates  of  programs  preparing  media  professionals  for 
the  level  tmst  ba  provided  a  reasonable  level  of  competency  in  the 
areas  of  traditional  librarianship,  traditional  education,  design 
(learner  analysis^  specification  of  objectives^  systems  design,  valida- 
tion, etGO#  and  production*     The  decision  as  to  whether  a  graduate  has 
such  competencies  should  not  be  based  solely  on  tb a  number  of  courses 
completed,  but  should  be  based  on  campetenc:y  t'^^^^ing.^ 

2,  The  faculty  of  the  programs  that     . -i?re  K^12  media  professionals 
should  initiate  programs  of  in-service  training  to  upgrade  the  competencies 
of  media  professionals  currently  in  the  field, 

3,  Efforts  must  be  made  to  foster  a  positive  attituQ^   l-?  Lhn 
building  principal.     This  can  be  done  thrQugh  semin  irs  at  the  meetings  of 
school  administrators  '  associations  and  through  puba^ishing  in  the  litera-- 
ture  that  reaches  these  administrators* 

4,  Local?  state,  and  national  organisations  must  continue  searching 
for  funds  to  provide  an  adequate  budget  on  which  to  baoe  an  instructional 
development  program* 

5,  Further  research  should  take  place  into  the  importance  of  an 
empathetic  personality  for  the  performance  of  instructional  develox^ment 
and  the  efficacy  of  empathy  training  in  affecting  personality.  Continued 
research  should  also  take  place  in  the  area  of  empathy  training. 

6-     The  media  professional  should  be  provided  with  at  least  one 
period  a  day  in  which  no  supervisor  tasks  are  required.     Such  a  period 
would  be  analogous  to  the  teachers-   "planning"'  period  . 


It  will  be  rerTiQmbored  that  the  number  of  cour^ej  had  a  much  weakei" 
relationship  with  instructional  development  than  the  competency  variables. 


7.  Finally,  K-12  media  professionals  should  be  provided  training 
in  personnel  management  to  allow  them  to  manage  volunteers  more  effec" 
tively. 

Continued  wDrk  on  the  development  of  instruments  to  measure  the 
^ouftt  and  quality  of  instructional  development  performed  at  the 
level  needs  to  be  undertaken.     Further  studies  should  be  performed  which 
utilize  observational  as  well  as  questionnaire  methodologiey ,     With  the 
einergence  of  information  pertaining  to  the  Gharacteristics  of  the  success 
ful  instructional  developer ^  perhaps  the  future  will  provide  us  with  the 
realization  of  the  promise  held  forth  in  the  literature— -each  media  profe 
sional  v/orXing  alongside  other  educators  toward  the  facilitation  of  the 
nurriculum* 
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Abstract 


The  Effects  of  Picture  Type  and  Picture  Location  on  Comprehend "^'^^n 
Phnip  J.  Brody  &  Alice  Legenza 
University  of  Kansas 

Using  ma themagenic-based  research  as  a  guide^  this  study  attenipted 
to  deterntine  if  the  location  of  a  picture  (pre-  or  post-reading  passage) 
or  the  type  of  picture  (overview  or  specific  incident)  could  affect 
reading  comprehension.    Ninety-two  conege  students  were  randomly 
assigned  to  one  of  the  cells  in  a  two-by-two  factorial  design  and 
required  to  read  a  text-like  passage  of  approximately  1200  words. 
Results  indicated  that  pictures  placed  after  the  passage  increased 
incidental  learning  more  than  pictures  placed  before  the  passage* 


Mathainagenics  have  been  studies  ^.ince  the  1950' s,  whan  Rothkopf  (1966) 
defined  the  term  to  refer  to  a  broad  array  of  behaviors  which  increase 
learning*    Although  mathemagenlcs  have  provided  a  theoretical  base  for 
systematic  research  in  some  areas  (Faw  and  Walker,  1976) ,  it  has  largely 
been  neglected  by  researchers  in  the  field  of  media. 

While  the  mathemagenic  effects  of  various  Instructional  techniques 
such  as  objectives  and  advance   organizers  have  been  studied^  the  most 
widely  and  systematically  studied  area  has  been  in  the  use  of  questions. 
Typically  (e.g.,  Frases  1S57),  in  these  studies^  students  are  asked  to 
read  passages  in  which  questions  have  been  inserted  either  before  or 
after  the  passage  (pre-  and  post-questions).    Students  are  then  given 
a  multiple  choice  exam  over  the  material.    The  results  of  these  studies 
generally  suggest  that  both  pre-  and  post-questions  facmtate  learning 
that  is  specific  to  the  questions  asked  while  post-questions  have  greater 
effect  on  learning  that  is  not  directly  related^  or  is  incidental ^  to  the 
questions. 

Research  which  has  attempted  tu  relate  mathemagenics  and  media  has 
generally  incorporated  pre-  and  post-questions  into  various  presentational 
formats  (e.g.,  Lavin,  1972).    Dayton  (1977),  while  describing  how  mathe- 
magenics can  be  used  in  the  design  of  slide/tape  presentationss  neglects 
to  discuss  the  important  impl cations  of  the  mathemagenic  hypothesis  on  the 
non-verbal  aspects  of  the  presentation. 

An  area  of  media  research  which  would  seeni  to  benefit  from  mathema- 
genics based  research  would  be  the  relationship  between  pictures  and 
reading  comprehension;  sonie  researchers  (e.g.,  Samuels,  1970;  Concannoiii  1975) 


suggest  that  pintures  m,;-'  -  -  c-. ert  with  comprehension s  while  others 
(e.g.,  Pr^ssley,  IS7}-;  imin  and  Lesgold,  1978)  suggest  that  the^  can  be 
beneficial.    Much  of  the  research  that  has  been  done  on  the  effscts  of 
pictures  on  comprehension  suffers  from  one  of  several  confounding 
variables.  Paeck  (1974)5  for  example,  utilized  materials  which  were 
not  representative  of  those  found  in  schools.    Other  research  has  either 
made  little  attempt  to  define  the  pictorial  attributes  which  were  being 
studied  or  has  done  so  in  terms  of  the  well-worn  realism  continuum 
(Dwyer,  1972).    How  then  can  mathemagenics  serve  as  a  guide  for  research 
in  this  area? 

It  would  seem,  first  of  alh  that  the  location  of  pictures  within 
a  reading  passage  could  be  of  iniportance.    It  needs  to  be  established 
if  pre-  and/or  post'-pi ctures  have  the  same  mathemagenic  functions  as 
pre-  and  post-questions.    Mathemagenic  research  would  seem  to  suggest 
that  pictures  placed  after  a  reading  passage  v    ^ -^.ve  a  greater  effect 
on  "incidentaV*  learning  (that  which  cannot  be  ie^rned  directly  from 
viewing  the  picture)  than  would  pictures  placed  before  the  passage; 
similarly,  pictures  which  provide  a  broad  overview  should  be  superior  to 
those  which  provide  a  narrow  focus*    Thus^  the  purpose  of  this  study  was  to 
1)  determine  the  extent  to  which  picture  location  and  picture  type  serve 
matheniagenic  functions  and  2)  to  determine  the  extent  to  which  these 
variables  affect  reading  comprehension. 


Metfiud 


Subjects 

Subjects  were  92  undergraduate  students  enrolled  in  education 
methods  courses* 

Materials 

A  passage  of  approximately  1200  words  about  the  square  of  Marrakesh 
was  constructed  from  a  variety  of  books  and  encyclopedias.    The  main 
theme  of  the  passage  was  the  daily  activities  of  the  square,  although 
half  the  article  discussed  history,  geography,  and  other  related  back- 
ground information. 

Two  black-and'-whi te  pictures  concerning  the  square  were  also  reproduced; 
the  first  picture,  a  suinmary  picturei  was  an  aerial  view  of  the  squarej 
showing  the  square  and  its  surroundings.    The  second  picture  showed  a 
close-up  of  a  specific  incident  taking  place  in  the  square^  a  snake  charmer 
with  his  snakes. 

Twenty-four  multiple  choice  questions  were  developed  over  the  passage. 
Half  of  the  questions  were  over  the  background  information^  while  the  other 
half  were  over  the  square  activities;  but  all  questions  assessed  incidentaU 
rather  than  intentional  *  learnings. 

Procedure 

Subjects  were  divided  into  4  groupSi  with  23  subjects  per  cell.  The 
groups  were  divided  on  tvvo  levels  of  picture  viewing  -  1)  before  reading 


the  passage,  and  2)  after  reading  the  entire  passage.    These  groups  were 
further  subdivided  so  that  half  of  the  subjects  within  wch  if^vel  of  the 
picture  viewing  saw  the  summary  picture  while  the  other  half  of  the  subjects 
saw  the  specific  incident  picture. 

On  the  day  of  the  experinient  each  subject  v/as  given  a  consent  form  to 
sign  which  stated  that  the  investigators  were  doing  research  on  media  and 
reading.    Subjects  were  randomly  given  booklets  containing  one  of  the 
pictures  1n  one  of  the  two  locations  and  told  to  *'study  each  page  of  tiie 
booklet  for  as  long  as  you  want  but  you  may  not  turn  back  to  any  page 
after  you  have  turned  to  the  next  page.    When  you  are  done,  raise  your 
hand  and  you'll  be  given  a  short  test  to  take." 

Rgsults  and  Discussion 

The  means  and  standard  deviations  of  correct  answers  on  the  compre- 
hension test  are  presented  in  Table  L 


Insert  Table  1  about  here 


As  indicated  in  Table  2,  picture  location  effects  were  significant  at  the 


Insert  Table  2  about  here 


.05  level       (IgSS)  -  4,22)  for  the  :.quare  activity  questions,  with  post= 
pictures  producing  higher  coiiiprehension  scores.    For  the  square  activity 
questions,  the  effects  of  picture  type  was  not  significant.    Neither  picture 
location  nor  picture  type  produced  significant  results  for  "background 
information  questions"  (see  Table  3). 


Insert  Table  3  about  her^u 


The  results  of  this  study  add  preliminary  support  to  the  idea  ..nat 
pictorial  attributes  can  serve  mathemagenic  functions.    As  with  "inserted 
questions"  reasearch,  this  study  found  location  to  be  a  significant  In- 
structional variable  since  pictures  placed  after  a  reading  passage  increased 
incidental  learning  more  than  pictures  placed  before  the  reading  passage. 
However,  the  lack  of  significant  re:-u;ts  related  to  the  "picture  type" 
variable  indicates  that  not  all  pictorial  attributes  affect  inathemagenic 
behaviors , 

These  mixed  results  call  for  additional  research  to  more  precisely 
determine  the  possible  mathemagenic  effects  of  pictorial  attributes. 
Most  importantly,  the  study  needs  to  be  replicated  with  different 
audiences,  particularly  with  younger  subjects  and/or  those  wi^li  different 
comprehension  skills  than  the  college  students  utilized.    Although  it 
seems  logical  to  assume  that  the  results  should  be  replicated  or  even  ■ 
magnified  when  using  younger  subjects,  results  of  research  with  both  reading 
behaviors  (Knafle  and  Legenza,  1978)  and  picture  utilization  behaviors 


(Pfessley,  1977)  differ  with  age  o\  ,'ihjects. 

Future  resenrch  (liust  also  focus  'joon  those  characteristics  which  will 
optimize  the  mathGinagGnic  effects  of  pictorial  attributes.    For  exaniple, 
the  optiniuin  length  of  the  passage  between  pictures  needs  to  be  determrnedi 
as  does  the  coniplexity  of  both  the  reading  passage  and  picture.  Other 
pictorial  attributes  that  affect  matheniageni c  behaviors  also  need  to  be 
identified  and  studied.    These  additional  attributes  should  be  applicable 
to  a  wide  range  of  educational  media,  rather  than  being  applicable  to 
pictures  only  (Salomon^  1978). 

To  the  extent  that  the  research  being  reported  is  replicated  and 
expanded  in  the  directions  indicated,  instructional  design  specialists 
as  well  as  all  other  classroom  teachers  will  possess  an  instructional 
technique  which  can  provide  substantial  benefits. 
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Moans  and  Standard  Deviations  on 

Correct  Ansv/ers  on  the  Coniprehonsi on  Test 


Treatment  Square  Activity  Background 

„  ^   X  sd  I  sd 

Pre  aridiifi  charrier  5.0  2.28  3.35  1.46 

Pre  square  activity  6.39        1.95  3.61  1.64 

Post  snake  channer  7.04        1.94  3.17  1.99 

Post  Gquare  activity  7.13        1.94  3.91  2.06 


Er|c  ^'  '  3S9 


Tab  la  2 


Analysis  of  Variance  for  ' -quare  Activity"  Questions 


L,:)cati  on 
Type 
L;  T 

Within 

To  tell 


SS 


elf 


10.27 
1.31 
.53 
381.05 
401.16 


1 
1 
1 

83 
91 


MS 

10.27 
1.31 
.53 
4.33 


4.2? 
.30 
.13 


■■()  <  .06 
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Analysis  of  Variance  for  BaLi.ijround  Inforntition  Questions 


Source  SS  if  M  £ 

Location  .10  1  .1  0 

Type  5.75  i  5.75  1.69 

LxT  1.31  1  1.31  .38 

Withiii  299.83  m  3.41 

Total  306,99  91 
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Abntrnc  t 


Two  meLlujcls  of  structuring  tcixtual  inatcrial  j  inornicitiun 
mapping  and  pvoyrrvimQd  inf^tractiDn ,  were  coin  pa  red  un  the  basis  of 
thoiv  ability  to  facilitate  recall  and  subsequGnt:  rei:rioval  of 
inf  0  rriuition.     Both  mCit]i.y\G  produced  very  significant  gain  scores^ 
however  no  differential  effects  occurred.     Retrieval  of  inforniation 
from  the  texj;  ^  a  taMk  independent  of  recall  behavior  and  based 
upon  di  iferont  *  ^dr^tuzl  search  strategics,  benefittad  significantly 
from  i  he  structural  cliaractcristics  of  the  mapping  technique,  A 
conceptual  baiJiB  for  the  issue  being  studied  was  suggested:  a 
comparison  of  systematically  inserted  adjunct  questions  and  explicit 
contc^nt  structuring  for  e liciting  mathemngenic  activities.  Sug- 
g,-^r"^jn:^  of  appropriate  literatures  i;uch  as  structural  linguistics 
are  liU.K'What  conjectural  because  of  the  absence  of  any  research  base 
for  iTiappiv^g* 


IjTt  r  o  d  u  c  t  ion 

A  rcvorsionnry  trend  coward  prinC,   textual  media  for  instructio!'  ia 
ba«ad  upon  Lha  increasing  importnncc  of  cost  effectiveness  as  a  selection 
criterinn.     Audiovisual  media,   in  varying  dGgrces  of  sophistication^  have 
failed  to  prouuco  Che  panaceas  predicted  with  cheir  development.  Couplocl 
with  the  cuntinuinB  prGeminancG  of  print  (verbal  niode)  instruccional 
mediaj   the  need  to  explore  the  potential  of  teKt  for  informat*    -  iub- 
mission  and  reLrieval  hao  nfi^ver  been  greater. 

One  of  thn  most  promising  print  methodologies,   information  mapping^ 
(MAP) J  will  be  compared  with  another  prominent  aethod  for  structuring 
prose,  programnied  itistruction  Cpl).     This  study  constitutes  a  seminal 
investigation,   aimed  at  documenting  main  effects  (evaluative  comparison) 
and  suggesting  some   CheoreCical  bases  for  considering  these  instruccional 
methods,     as  such,   it  should  raise  more  questions  than  it  answers,  however, 
this  seems  appropriace  considering  the  embryonic  state  of  information 
mapping  research. 

Prof^ rammed  Ins  true  t  ion 

Programming  textual  materials  normally  consists  of  limiting  the  size 
of  information  chunks,   re^quiring  overt  responses  to  the  informacion 
prescntGd,  providing  feedback  (reinforcemenC ) ,   and  allowing  for  indi-- 
vidual  pacing.       Ev/iluative  research  or  pror^ramm  instruction  abounds, 
the  reviews     (Lango,   1972;     Schramm,  1964^     /enerally  indicating  a 
slight  learning  advantage  and  a  more  si.jnificaat  time  savings  for  PI 
when  ''^ompared  with  traditional  instruction.     In  analyzing  the  contribu- 


Ir*  .  rusaLiOn  Mapping  '    is  a  trademark  of  Information  Resources,  Inc. 


tions  of  the  a  foremen  tionad  characteristics  of  PI,   Uartley  (i  974)  con''    '  ' 
that  nonti  were*  easentic'il  to  Lli'i  succriss  of  any  prograin,   thcit  removing  one 
or  more  of  Chesci  techniques  has  Little  if  any  effGct  on  the  learning 
rosiil  t'S . 

The  intention  of  study  is  to  identify  a  common  basis  for 

comparing  media j  not  'citing  research  on  programming  attributes,  'rhcj 

programmed  vermin'      i    icKC  ]."cpared  for  this  study  could  alternatively 
ba  de  fined  as  th-   jiyn^  ^-atic  insertion  of  adjunct  questions  (Rothkopfj 
1972)   into  textual  :.\-iVvr\al,   as  multiple  choice  questions  wore  consistently 
inserted  after  chunks  of  prosG^  with  knowladge  of  results  presented  immod-' 
Lately,     IVhile   the  programmed  material  compared  with  mapping  in  this 
study  includes  all  of  the  characteriritics  of  programm.  - cited  above,  it 
could  easily  be  sumsumed  by  another  rubric.     Tiie  naming  of  the  technique 
is  not  as  important  as  the  underlying  cognitive  processeg. 

Tlie  issue  of  importanca  is  the  degree  to  which  consistent  overt 
responding  to  textual  material  v;ith  subsequent  feedback  evoke  recall 

behavior.     Several  s     -ies  hav^^   supported  the  facilitative  effect  of 
inserted  ;  ^*s  u^-que  s  t  iou^ ,  espacially  for  intentional  l^'arningj  on  immed- 
iate postrists  (Frasc,   1968;  Rothkopf,   1972;   RDthkopf  &  Bih^Fcos,  1967). 
Learning  can  be  expected  to  improve  as  a  function  of  the  programming 
technique  employed  in  this  study. 

Information  Happing 

Information  mapping  is  a  technique  for  developing  textual 
materials  developed  by  Robert  Horn  almost  a  decade  ago  (horn,  Nicole 
and  Kleinmans  1969).     It  consists  of  a  specifi:  set  of  rules  and  pro- 
cedures for  classifying^  organiHingj  and  presenting  information.  Tlie 
primary  characteristi::s  include  chunking  information  into  units  and 
presenting  it  in  labelled  blocks,   the  marginal  labfslling  providing  an 


intrinsic  indexing  Bye  tern.     With  the  prancntation  format:  raiuainlng 
conHistent,   tfv^^c'a  inCorniation  blauks  are  saquericed  congruously  v^i  l:h  tlu' 
naturu  of  the  infonnation  being  presented  into  superordi Ji.atc  structuroii 
or  in''3ps   ( s  t  rfic  turc  ,  procedurGj   procesSj   ffjct^  concept  and  principal 
lypGS).     The  structurai  clmrcic  tori  sties  DKternally  organise  infonnation. 
with  hypothesizGd  f  aci   L  ta  tion  of  information  coding  and  retrieval* 

As  an  ins  t  rue  t  ion  a  1  moth.ode^lagy  j   i  nf  anna  t  ion  mapping  is  only  boginni 
to  omcrge.     Used  as  forniat  for  prcKcnting  fonnal  reports   (Hart jcn  ,  1977)5 
it  has  boeri  rGComrntnded  as  a  valuable   technique  for  instruction 
(Thiagaragan  J   1977  ) ,   industrial  training  (Horns  1975),  computer  ns  si  as  ted 
instruction  (Horn,   Nicolj  Ronianj   &  Razarj   1971)  and  task  analyr»is 
(Shroedcrj   1975  )*     Surprisin;;-;ly  litUlG  literaruic  has  chronicled  its 
developmGnt  as  an  Important  inediurn  for  industrial  communication*  Even 
less  empirical  analysis  oxistSj  although  research  i$  currently  in  progre^- 
espGcially  in  England,     Tlic  %wo  most  prominent  studies  failed  to  documeuK 
any  learning  benefits  frcm  iiU.jjping  when  comapred  with  proHC  (Horn,  Nicnl, 
&  Kleinman,    1969)  or  wit!:  ^.ym\.r:tQr  aasis'    d  instruction  (Geiscrt,  1970)^ 
however  attitudes  toward  niap.;^  m    a^:  ^  3  '  i:i  i  ^  superior  in  both.     However  j 
neither  study  provider  any  theorfcitical  or  empirical  support  for  the 
claims  of  its  developer, 

As  a  seminal  effort,   this  stu'v  focused  on  the  potential  of  some  of 
the  major  characteristics  of  prograirjiied  and  mapped  text.-j  ror  eliciting 
recall  and  as  a  medium  tor  information  retrieval.     Conceptual  foundation^a 
wilt  De  considered  later. 

Roc  a 1 1  *     Since  no  analysis  or  manipulation  of  mapping  variables  has 
been  doncj  inferences  or  predictions  about  .Mapping  charactoristics  must 
derive  from  other  literatures,   specifically  that  relntod  to  the  •^omantLc 


structure  C3£  prose  J  an  issuo  i^'hich  has  recciivccl  con^idcrabla  q LLont: iijii . 
Prose  passn^'.os  nom^illy  conUnin  an  implicit  orgL-  Li^ationnl  stnteturt"  of 
contont.     Ilecall  of  ideas  bucomes  a  function  of  thu  hcigbf:  in  the  content 
structure  of  a  given  idea,   i.u,,    idcias  high  in  the  st;ructure  aro  better 
recalled  (Hayer  &  McConkie  ,  1973).     This  ongcndered  thei  concept  of 
staging  (CriiiiGs.   1972)5   the  dimenRion  of  proso  structure  that  dctormim^s 
the  prominence  o£  ideas  within  prose.     ClGmonts  (1976)  found  than  prose 
segments  a*:  high  staging  levels  v/ere  recalled  better  ^ 

This  typo  of  rGscarch  convinced  Meyer  (1975)   that  she  could  "ijn/^''  !" 
this  structure  to  the  learner  by  adding  non/-content  signals   ,/''.^he^%>  v. 
two  approaches.     One  is   .    *    .    ■')  that  would  emphasiER  aspc.  .h    ji  the 
content  or  its  structure.     Improved  recall  resulted^  however  additionnl 
research  is  needed.     This  type  of  research  sets  a  precedent  for  the 
extrinsic  reporting  of  prose  structure  ,  as  occurs  in  information  mappiiigj 
where  types  and  levels  of  infoinnation  are   labelled  allowing  the  learner 
to  relate  aspects  of  content  together  using  the  tr:hniques  of  blocking 
and  labc  1  ling. 

The  blocking  of  infonnatiGn  produces  the  effect  of  associating 
linguistic  elements  together  in  a  holistic  fashion*     Informatier;  ^'nclosoJ 
in  a  block  becomes  a  unitj  permitting  comparison  of  wi thin-block  semantic 
s imi lar i tiei'  to  differences  in  elements  presented  in  other  blocks  (Groprjerj 
1970),     This  chunking  of  infonnation  can  also  be  reasonably  expected  to 
facilitate  encoding  of  locative  infoinnation  about  pas  sages  into  nieniory. 
Christie  and  Just  (19  76)  corroborated  earlier  results  in  concluding  chaL 
individuals  accurately  remember  tlic  location  of  inforvnarion  in  tcKC. 
Those  incidental  locators  can  act  as  spatial  attributes  of  memory 
( Un d 0  r wo o d  ,   1969)  wh  i c  h  f  u n  c  t  i  o n  to  d  i  s c  r i  m i  n n  t  u  one  tiiem o  r y  f  r im  0 no  t h  e r  , 
thereby  facilitating  tctrioval  of  the  aHHOciated  infomation  frDin 
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memory.     The  extriusic  hl.ocking  of  inCorination  can  only  ba  GXpocted 
to  strengthen  thcBC  spnUial  cufiSj   facilitating  recoil  of  informatiou 
int{?^c . 

Predictions  about  labelling  must  bcj  derived  from  tliG  liLuiraturo  on 
hGadings.     Functioning  as  advanced  organizers ^  headings  provide  concoptual 
cues  about  the  ansuing  information  in  a  passages  providing  an  '^explicit 
struCtUTe  v/hich  aBsists   the  reader  in  integrating  the  inforniation  as  ho 
reads"  (Wright^   1977,         9G).     Recall  has  been  facilitated  by  subsumliig 
information  under  major  Bubheading3   (Dahlj   1973)  or  by  presenting  ona 
line  titles  immediately  prior  to  the  passage  (Dooling  aiid  Lachrnan,  1971^, 
Wright  (1977)  recoinmended  differentiating  the^^-2  headings  from  the  body  of 
the   texts   either  spatially  or  typography  ^  as  is  done  in  infomation 
mapping. 

Retrieval.     Since  information  mapping  has  been  recommended  as  an 
effective  tool ^  consideration  of  the  information  rotrieval  characteristics 
seem  ImporLant,     Howeverj  the  literature  related  to  information  reLrieval 
is  concerned  either  withmomory  pLJces&es  or  automoted  retrieval  systems 
modelled  after  the  fornier.     Techniques  for  facilitating  the  retrieval 
of  information  from  text  materialj   such  as  those  suggetTted  by  information 
mapping  (table  of  contents  j  predict:  '.>le  textual  for  mat  ^  consistent 
typography,  marginal  labels^  and  local  indexing)  operate  in  ait  atlieorc tica  1 
context  J   BO  that  any  successes  forthcoming  must  be  explained  by  other 
literature.     Searching  is  generally  agreed  to  be  aided  by  section  headings 
presented  in  the  text  and  in  a  c  or  re  s  ponding  table  of  con  ten  I:  s  (Wrightg 
19  77)  J   but  how  these  and  other  techniques  work  is  uncertain. 

Textual  searching  becomes  a  matter  of  associating  locative  informa- 
tion (spatial  cues)  stored  in  memory  with  their  referent  as  it  appears  in 
a  passage.     Infoniialion  mapping  nrovides  explicit  cues   (  labols)  wh.ich 
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may  ba  ciiccKlcid  ^ih  louativc  Informntian  iii  niatiiory.     In Coriniition  rotricvnl 
'  requires  a *isaciat ini^  only  thfi  locntivc  cwo  v^itli  t\m  labal  j  tharoby 
reducing  meinory  loncl.     At  thu  very  least  ^   libGls  may  be  uaod  as  a  con-- 
veniunt  tag  for  Bcjnrc-hitig  toxt. 

Some  iniplications  of  blocking  also  st^ ciii  obvious ,     Christie  and  Jur.t 
(1976)  hypotliO riizod  that  "locatives  infornin  ti  oii  could  provide  an  inclox  to 
tVie  spatial  cliBtribution  of  inf pniiatiori  in  t  lie  possage*'  (p.   706).  Sinco 
re tr Laval  of  informntion  from  a  pcis^tagDj  mea     tgcI  by  eye  fixations  ^ 
more  effitiiont  for  organiKcd  pasBagfiSj  explicitly  chunking  information 
into  units  can  Dnly  be  inferrad  to  provide  h  tronger  or  mox^e  easily 
.    cliffGrcMitiab  le  locacivQ  cues*     ViHuaL  sc-mnning  should  be  accomplished 
more  crisilyj  Che  re  by  increaHing  tG^tiial  tetrieval  capabilities. 

Finally  ,  the  consistent  format  f orfteGquciiciiig  information  Buggests 
a  iearcb  strategy  for  locating  in forination .     Search  rule|  arc  suggested 
by  the  rnatarials  bccouse  the  iGarncr  Itaows  conceptually  wh(3re  a  particular 
dttribute  is  locaf^ed  in  relation  to  other  niacfirial.     This  eKplicit 
sequencing  aboulcl  provide  the  basis  for  a  corisistent  search  procedure. 

Method 

Jjirtici  pants 

FcDrty^onc  seniors  and  graduate  students  enrolled  in  two  sectiDus 
cf  an  introductory-  media  course  were  4issigned  to  treactnent.  Attrition 
accounted  for  unequal  N'a  (MAP  =  22,  PI  *  193.     Verbal  ability  level 
and  Subject  background  wetre  assunied  equivaLernt  as  no  significant  differences 
fee  twee n  groups  in  pretest  perOormance  occurred* 

F  r  o  c_e  d  u  r  e^s 

All  subjects  vcre  prates  tod  and  aBGi|iiicc£  to  trGeitment ,     One  we  ok 
following  the  pretCBt,  the  nc  1  f^-^.s tudy  nunt^rials  ware  presented  to  the 
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participants,  who  ware  allowod  to  compleCF^  thoni  at  LliDiir  own  paca. 
IriingciiatGly  fnl  lowitif/,  completion  of  the  Cc-xt,  each  indiviclunl  turned  in 
the  matorial  and  requested  the  aKani  (sanie  as  prcitcist)^  a  AS-itonij  four'- 
optiDii  multiplo  ciioicfi  GKQin,     Test  ilGms  worG  oxtraCKGd  verba tim  frcini 
the  text.     The  test^H,  in  addition  to  the  tCKta^  were  reviewed  for 
content  validity  hy  three  subject  matter  OKpt^rcs,     Reliability  of  thu 
inBtrumont  was  oBtablisjhed  by  Lhc  Spcarman^Brown  Hplit--half  tochniquo 
(r  ^  .83). 

Two  weeks  following  treatment  all  subjects  were  administered  the 
re trieval .  examina ti orii  an  open^bDOlc  task  consisting  of  41  sentencG-- 
completion  quGstiona  that  '/'equired  reference  to  the  textual  macerial,, 
A  total  of  one  hour  was  allowed  for  this  eKam  for  all  paTticipantB ,  To 
insure  retrieval  from  the  CQxt  occurred  rathor  than  rocall  behavior, 
subjects  were  required  to  indicate  the  page  nuiiiber  in  the  text  that 
contained  the  answer, 

A  pretest  -  post  test  design  v/aa  employed  iri  this  study,  with  each 
treatment  group  serving  as  the  control  for  the  other  group.     In  subse^ 
quent  studies  a  controL  group  (narrative  pross  test)  vill  be  employed. 
The  delayed  test  was  not  intended  to  assess  the  same  diinensiori  of 
cognitive  functioning  as  the  recall  (pre tes t-^pQStte st )  task,   so  those 
rGSults  were  analyged  Beparacely, 

Results 

A  2x2  analysis  of  variance  (ANOVA)  (test  X  method)  was  calculated 
for  the  data  presented  in  Table  1,     A  significcint  main  effect  for  tre^t- 
ment,  F  (1,   39)  -  340.92,  £<.0Ol,  %s?as  DKperienced  by  both  groups  ^  though 

Insert  Table   1  about  hcr^c 
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no  clif fartuititil  affect  oi  trariCmcint  (grcmp  K  to&t)  ouciirre^^lj  as  gciln 
MCorcs  wore  fairly  equivalent,     Sinco  no  ai  gni  f  i  c^int:  cli  f  forcncc^s  in 
prcUest  scores  and  poHttost  scorus  occurrodj   it  ccin  bo  concluded  tluit 
both  the  HAP  treatnKjnt  ancl  the  PI  creatinine  pi^oducecl  Bimilrir  amounts 
of  learning. 

An  F  test  tot  honiogenciity  of  varidncG  in  tbo  retrieval  tost  scnrcs 
indicated  equivalcnice  (F  (21,   18)  ^  l.Ui   p>.05)  sn  Cliat  a  t^  test  for 
indopendant  rneaiiH  Vv^as  conducted,     A  sig^nificant  ciiffercnce  Ct_  (21^   18)  * 
4.14j  p<'.00i)  botween  tho  scores  of  tliG  trQatiTunit  gToupB  accurredj 
indicating  that  tliG  MAP  group  porfornicd  substun tially  better  than  tho 
programnied  group  on  the  retrieval  task.     It  is  obvious  from  the  data 
that  the  rGtrieval  task  was  differGnt  and  independent  from  the  earlior 
recall  task.     The  relationship  between  posttesc  scores  and  rGtrieval 
scores  was  axtrcmDly  low  and  insignificanc  (r  »  .15)  and  the  variances 
for  the  retrieval  task  were  substantially  highet*     This  indicates  Chat 
chis  task  was  not  facilitated  by  earlier  learning  (Tec  all)*     In  an  attcinipt 
to  control  for  the  effects  of  prior  leainningi  ati  analysis  of  covariancG 
was  conducted  i  indicating  that  recall  (ptior  learnitig)  was  not  a  signifi^ 
cant  covariate  with  retrieval^  confirTning  the  independent  iiature  of  the 
retrieval  task  and  the  superiority  ot  the  MP  txeatnient  for  facilicating 
it. 

D  i j  c  us  J  ion 

Rgcall 

Significant  iG^rning  (recall)  Resulted  from  both  the  PI  and  MAP 
techniques 5   indicating  that  for  initial  leafnlngj  both  are  adequate  as 
presentatianal  media.     Without  a  control  group  trQAtincnt  (narrative 
prose),   the  c^^tcnt  of  recall  fncilitatipn  rfHulting  f rom  Chf'ir  rcapcctivo 
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charQCCcrint  icfi  rcmainB  undocuincntud,     It  can  only  be  concludotl  ulicy 
produce  Gciuivainnt:  nmounts  of  Icnrning,     While  ini  t i nl  ]y  cli Rappoi.  liL  i  n);., 
tbi  K  cquiviileiice  in  raKultant  loflrnini^  pravidcB  a  ma^iniiigf ul  einpi  cal 
contO}<c  for  guggoH fling  a  concnptual  bii^rhi  Cor  tliolr  compnri san . 

The  advaiitagOB  of  mapping  arci  obviou£ily  Btfuctural  ,  bciBecl  on  the. 
prGiiiise  tliat  by  breaking  clnv;n  infonnatian  inco  small  chunkB  and 
explicitly  structuring  the  contcKt j   learning  will  bo  impraved.  FraBc 
(19G9)  provides  Grnpirical  support  concluding  that  recall  is  enlinnced  by 
informing  the  rendar  of  the  structure  of  a  paBsagQ  prior  to  roading  thG 
motGrial.     By  cloarly  defining  the  function  of  the  bo  in  fortnati  on  chunks 
and  their  conceptual  relationship  to  other  chunks j  you  are  explicitly 
organising  material  for  memory.     Like  advanced  organiE^rs  (Ausubalj  1968), 
mapping  provides  conceptual  tags  which  function  as  an  explicit  ideational 
scaffDlciiTig"  for  aaaimilating  informatiQn,     McCDnkie   (1978)  suggGsts  that 
a  complete  understanding  of  the  effecCB  of  prose  structure  on  recall 
probably  necessitates  looking  at  some  strucCural  linguistic  literature 
(Grimes,   19  7Z;  Meyer  ^   1975),     For  purpoaes  of  this  diseueaions  stiffice  it 
to  say  Chat  the  iinportant  intrinsic  characteristic  of  mapping  is  the 
structuring  of  content  in  teKtual  passages  *     A  meaningful  conteJct  for 
comparisig  mapping  and  prograniiTiing  is  the  mathOTagenic  hypothesis  . 

The  concept  of  mathemagenic  activities  (Eothkopf j   19  70)  should  be 
familiax  to  most  researchers  *     Meaningful  prose  learning  has  been  con^ 
sistsntly  enhanced  by  the  elicitation  of  mathanagenic  activities  in  the 
learner  5  the  covert  mcdiational  behaviors  that  produce  learningj  that 
determine  "the  natu^re  of  the  effective  stijTiuljL  i^ti  cxperimenCal  oi^ 
in  s,t  r  u  jc  i  on  a  1  situations"     (Rothkopfj  1970,  p.   3265  emphasis  his). 
Several  techniques  have  been  rcviov;cd  aB  capablo  of  manipulating  loarncrs 
cognitive  activitien  during  ncquifi  i  tion ,  Huch  a  a  advance  organip.crsj 
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rasponHO  modes  (untlcclinini^ j  tuitctakinn)  j  objuctivcsj  and  iiisortcid 
quostionn  (Paw  ^  WnUorj   1975),     This  lint  nhauld  innluclo  ojtplicit: 
content  fil:ructuirin[3  that  is  provLclecl  by  informoliion  mapping  procucluroB, 
It  can  be  tQntfjtlvaly  eancluclcd  that  this  tuchniquc  dirnctly  aftoct?) 
Class  111  nia thauiagciiic  activiuiGOj   thoHG  conccriicd  with  visiinl  Bcaniring, 
trnnslation  intn  mp^iningful  constructRj  and  intcrn^il  cognitive  procasBin^i; 
(di scr iminat ion  1  di  fCcrcmtintion)  of  prose  (Rothkupfj   1970).     It  HOcnis 
obvious  that  ordering  and  prenrranging  teKt  wLll  affoct  rending  as  do 
the  other  tocbniciuog  siiggestcd  earliGr,     So  the  issue  actually  addrGssed 
in  this  study  concQxus  the  type  and  extent  of  rnathsmagcnic  activities 
olicitGd  by  consistently  inserted  questions  vis  a  vie  eHplicit  content 
structuring.     At  the  processing  stage  o£  Clafis  III  behavior ^  aoniG  form 
of  organization  of  material  inevitably  occurs  that  will  facilitate 
metnory  coding.     The  structuring  implicit  in  the  mapping  technique  could 
bo  hypochGai^ed  to  supplant  these  mathemagGnie  activities  (Ausburn  & 
Ausburn,  1977),     A.t  any  rate,  it  seems  logical  that  itiathemagenic  activity 
is  a  meaningful  cotiCext  for  comparing  and  analysing  the  effects  of  these 
instructional  methoda  on  learning  (recall)* 
Re  tf ieval 

The  structural  characteristics  of  mapping  provide  a  clear  advantage 
for  retrieving  information  from  textual  materials.     Sonie  combination  of 
the  marginal  labels^  consistent  fomiatj  and  infonnation  blocking  signifi^ 
cantly  facilitated  the  visual  search  procedure.     In  order  to  understand 
the  contributions  of  eachj  these  variables  will  have  to  be  manipulated. 
It  should  be  obvious  from  the  previous  discussion  of  matheniagenic  activitic 
that  an  alternative  construct  is  needed  to  adequately  deal  with  the 
retrieval  process^   since  learning  is  not  the  issue  of  cnnccrn.  Rather 
we  need  to  be  concerned  with  those  textual  characteristics  that  will 


maximally  facilitfltc  tlic  aicnrcli  for  specif ic  inf oiiiioti on .     A  rcvicv.'  of 
the  rcfcrcmco  nncl  incloKing  litmraturc  should  bo  combinocl  vntli  an  annlyBis 
of  autumatecl  in  forinatiaii  rotriaval  fiiyBLomo  filtered  througli  n  viBual 
infonnation  prDceBsing  porspcctivej  tmch  as  tlmt  suggcstod  by  Clark, 
CarpenCGr,  and  Jugt  (1973),     The  retrieval  iBsue        important  and 
diatinctive  enougli  Co  bo  divorced  from  consideration  of  the  learning 
potential  of  mapping,  since  diftGrent  asBumptions  and  analyses  will  be 
ra  qui  red  for  iti;  proper  consideration* 
Surnin^ry 

The  potential  of  in f onnati on  niapping  as  an  inBtructional  technique 
sGfims  obvious,  however  the  virtual  absence  of  any  research  baee  relegates 
mapping  to  the  "potential"  status.     While  this  study  was  metliodologically 
deficient  (absence  of  control  group  ^  study  tiTiic  factor^  prior  experiGncej 
or  concern  with  relevant  learner  aptitudes),  it  was  intended  only  to 
globally  address  the  issues  and  assumptions  of  mapping*     The  need  for 
additional  research  to  consider  the  level  and  type  of  learning  and  inter- 
ections  with  the  characteristics  of  niapping  and  programming  is  unlimited. 
Additional  research  can  be  expected  to  clarify  the  usable  parameters 
of  both  methods  of  instruction  as  well  as  shedding  another  glimmer 
of  light  on  how  learning  occurs. 
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Table  1 

Mcana  and  Daviations  for  TroatmcnC 
Groups  by  Test 


Pre  tc  St 
(Recall) 


Map  Croup 
X  28.18 


SD 


3.  51 


Programnied  Ins trtictiqn  Group 
27.84 

5.37 


Pos  ttes  t 
(Recall) 


SD 


38.85 


2.87 


37.53 
3.37 


Re  tricval 
Tost 


X 
SD 


33.90 
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The  Instructional  Effectiveness  of  Integrating 
Abstract  and  Realistic  Visualization 


John  H,  Joseph 
The  Pennsylvania  State  University 


The  relative  effectiveness  of  abstract  and  realistic 
visualization  is  debated  in  the  literature  and  by  practitioners. 
The  study  examined  the  instructional  effectiveness  of  integrating 
abstract  and  realistic  visualiziation  for  self-paGed  and  externally- 
paced  instruction  involving  five  types  of  instructional  tasks 
and  students  of  different  general  ability , 

The  results  suggest  that  designers  of  visualized  instruction 
may  expect  to  enhance  its  effectiveness  under  general  conditions 
by  inclusion  of  realistic  visualization,     Howeverg  the  inclusion 
of  abstract  visualization  or  the  integration  of  abstract  and 
realistic  visualization  should  be  based  on  a  Gonsidera1-ion  of 
pacings  general  ability 5  and  the  type  of  instructional  objective. 
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The  Instructional  Effectiveness  of  Integrating 
Abstract  and  Realistic  Visualization 


John  H,  Joseph 
The  Pennsylvania  State  University 


Paper  presented  at  the  197  9  Annual  Convention 
Association  for  Educational  Conununication  and  Technology 
Research  and  Theory  Division 

Me'w  Orleans 3  Louisiana  March  7^  1979 


Thl^  study   iKamlnid  ttie  qaesirion  of  ^h^ther'  t^e  ^  integr'a-tion 
of  alstract  and  realistic  vi^Uiliaa^ion  qb^ji  iinprove  the  effective^ 
jiess  of  \risaializid  lustpiac^icPn,     The  questiDn  is  derived  from  the 
peTeanni-al-  issue  of  the  relative  v?or"th  of  realistic  and  abstrac-t 
visuals,     VJJether  Tealitj^j   with  i'ts  myriad  oi  onms  wh  icli  aipa  not 
dixectLy  relevant        a  par-tii^ular  leaarnijig  olj  ec-tive^   is  a  more 
efJective  aid  to  comjnuiiica-Cl^n  ^han  ■some  edited  3Depre  seJitation  of 
reality"  is  still  debattd        pmcitAtioners  sad  i^e sa^rc Oleics  • 

The  serier^al  prefe^^eJice  for  reaiisJn  in  edncaticnall  practice 
Is  reinfoTced  by  several  tj^ecy^ists.     Dale^s   (19HB)   "Cone  of 
Exper'ieiic  €"  h&s  been  particular 3-y   influeiitlal.     Cat^pe  Jiter 's 
<1953)    "sign  Similarity  Sypo-thesis"  regarding  fiLjn  ef dfectiv^eness 
and  Morris*    (194=6)    "Icondcitjr  Tlnaory^"  ape  also  supportive  of  the 
notion,     Iwyer'  refers        thest         "r^ealisni  tlieorieSi'- 

Th^  jreferenoe  is  als^  bts^d  partly  on  "tha  affective  value 
of  realistic  visuals*     TJ^at  is^    the^^  are  e^pacted.  to  srouse 
in'tearsst  and  gain  and  sustain  ai:t©ntioii*     Thie  irttu^itive  basis 
is  supported  some^Jnat  by  firiU  Iu^b  of  a  genersl  pr^ef  erence  for 
compiex  detailed  iXlus'tr^ti  on       part icularly  by  ^ider  clmild^eTi 
and  aduats  CFrench^  1952  ;  Spa^lding,  19S5;   Hanes^  l  973). 
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Dember  ilscussed  basic  Jes^arch  which  aLso  i\iggists 

tlnat:  compli5<i^y  is  4  "mjoan  deter mtmiit  of  attention  and  s^plora-^ 
t J,Dn  in  botli  &nLina.ls  a^d  man'*  Cp^  ?SZ).     ^^rb-us   (ISS  7)  5  Kahnenian 
(i973>j   and  H&ber  (1973)   cit^  cpeLated  tvidenca  th^t  coJnplt^c 
areas   of  %^imum^l%  4^tr^act    &ttenttoj% ,     Deinbe^"  C 1960)  aLso  pMvideg 
i^idence  that  st  la^lus  compl^^tty^  coiistitiates  a  rem^aing  condi- 
tion. 

a3^temat iv'eLy^  tJia^t  ^n  increase  tn  xe^lism  ^ill  not  iitcesssriljr 
lead  to  i^crease^  D.e^ming.     Tbese  la^if^  r^eseaTchers  fail  ttat 
tKe   irrelevant  Qtjes  ina^  Inter ifen  witJi  iiaming.     Trovers  C19  61+ ) 
advocates  that  ttie  discarding  of  lisi  irfiportant  ih^fo^nistion 
and  th€  xeteM ion  of  jnore  Impo^t^jit   sjiould  be  dDTi%  for  tfie 
leariier  to  asswa  effcatlv&ness   (pp^  380-3  82  ).       Qlbsori  ^1^6© s 
that  line  drawings  render  the  mos^  iinporta^it  property ie s 

of  an  objects  ''tine  foTu^  shapes   ajid  pr^opop-tlon  oJ  its  edgag  and 
suxfacas,'^    He  sugges-ts  tha±s   "3f  ^0  assme  that  a  pmva^iy&v 
do«s  not  need  -to  be  given  ^Xl  tjia  piroperttes  of  an  absent  pbj  ect 
in  order  to  Knc?w  how  "to  deal  vqx±]\   it;  5  bu^  only  those  whici  a^e 
relevant  or  signL^cjrit  ^  it   is  a  v^a^t©  c>J  effort  ta  simuLate 
theia  air'  ClSSLf,   pp.  21-22).     Hochbprg  ^Iso  suggests  that  ^^the 
chspactaristic^  of  a  given  object  Tn^y  be  conOTuniaa'ted  bet^ei  as 
the  r'epreientatlo nal  fidelity  of  'ehe  sij^arogata  deteriorates 
(1962,   p*   30>,   D^oeJ^  C1972)   fpur^d  e\ri4€n^e   to  support  this 
notion  und#r  cert^iJi  condlticris. 


Bx&adbent  (19  58^  1965)  and  Travers  (19  6^^,  1970)  see  this 
Inability  to  handle  e^ccesB  detail  as  a  func^Lori  of  the  limited 
capacity  of  the  human  information  processing  system.  Broadbent 
diSGrifces  a  system  in  v^hich  irrelevant  iiiforjnatlon  is  "filtered" 
through  selective  attention. 

Dwycp  found  that  the  type  of  vis\ial  illu^stpatlon  C abstract 
vs   i^eaXistic)  i^hich  was  most  effective  for  Instpuctiori  was 
depejident  upon  the  type  of  inforjuation  to  hm  transmitted  ioT^ 
ihm  kind  of  objective) ,  whether  the  instructiori  i^as  Belf^-paced 
Kir   externally  pacedj  the  grade  levels  and  the  level  of  entering 
JehaviQr  of  the  students  (1972)*     The  type  of  illustration  most 
-effective  was  also  found  to  be  dependent  oti  tl\&  level  of  general 
ability  CIQ)   (1976),     For  certain  ediacatioiial  DbjectiveSs  Dwyer 
^oujid  Tto  signif icaiit  differences  betweer  verbal  only  and 
^is  tialt  zed  tr eatmerit  s  (19  72  ), 

The  purpose  of  this  -study  was  to  in^/astigate  the  possibility 
^ha^  presentations  which  integrate  realdstlQ  and  abstract 
vis uali mtion  might  prove  effective  under  Kid re  general  conditions 
^itm  respect  to  the  learner ^  content  and  pt^e sentation .     Two  methods 

integnating  the  visiialisation  were  Investigated-       The  first 
j^itlnod  ^as  with  the  hybrid  illustration a  real  color  photograph 
i^nto  which  a  line  drawing  segment  has  been  iriserted.     This  provided 

abstract  illustration  of  the  important  ralevant  elements  to 
p^e  conunun icated 5  as  well  as  an  arousing  realistic  content*  The 
0.econd  nnethod  was  through  the  simultaneous  presentation  of  both  a 
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line  dra^^lng  and  a  real  color  photograph.     In  tTiis  treatment  the 
abstraot  line  drawing  could  communicate  the  essential  information 
and/oT  facilitate  looking  at  the  fealistic  ii.lustration.  The 
realistic  illustration^  on  the  other  hand,  could  motivate  and 
arouse  as  well  as  coimnunicate  essential  information  and  i^elate  it 
to  r^eali-ty.       The  effectiveness  of  these  treatments  was  measured 
for  various  kinds  of  learning  objectives 5  different  general  learning 
ability  5  externally-paced  and  self-paced  methods  of  instruction ^ 
and  inmediate  and  delayed  retention* 

Procadure 

Subjects  were  490  -tenth  grade  public  school  students  enrolled 
in  mndatory  health  classes.     Classes  were  coeducational  and 
there  warn  no  ability  grouping.  The  subjects  were  stratified  into 
high  5  medium  and  low  general  ability  groups  on  the  basis  of  their 
scores  from  the  Analysis  of  Learning  Potential  test.     All  subjects 
received  a  general  physiology  pretest  prior  to  inGtruction.  The 
inetj^uctional  unit  and  criterion  tests  were  those  developed;  and 
used  by  D^yer  (1972).     The  25OOO  word  instructional  script  deals 
with  the  construction  and  operation  of  the  heart  and  is  accompanied 
by  37  visiialSj  designed  to  illustrate  concepts  and  relationships 
for  which  visualization  is  likely  to  be  beneficialp 

The  criterion  tests  were  a  drawing  test  with  18  items  and 
identification 5  terininology  and  comprehension  tests  with  20  items 
each*     Together  these  four  tests  constituted  a  total  criterial 
test*       Reliabilities  of  the  individual  tests  equal  or 


exceed  .76,  The  reliability  of  the  total  test  is  ,91,  according  - 
to  Dwyer  (1972,  p,  12), 

Half  of  the  subjects'  received  self-pac^d  writteri  instructipn 
and  half  peQeived  eKternally-paced  instruction  by  an  audio  tape 
recording  (with  visuals  in  booklet  form)  .     WltJiiri  each  of  these 
two  modes  the  students  received  oiie  of  the  five  tre4tnientss 
in  struct  ion  with  simple  line  drawings  ^  instruction  i^dth  rfealistio 
color  photographs  (3X5  inches) ^  instruction  with  both  line  draw^ 
ings  and  color  photographs,  instruction  with  hybrid  dllustrations 
of  realistic  photograph  and  line  drawing  segments ^  and  a  control 
group  with  no  visuais*     Treatments  were  randomly  assigned  to 
subjects.      All  student^  received  the  four^part  criterion  test  on 
the  day  immediately  following  the  instruction  and  again  after 
four  weeks. 

The  two  methods  of  integrating  ahstract  and  reaListic 
visualization  were  each  separately  coinpared  with  the  oontrol  groups 
the  realisLic  visualization  treatment  g^'owp  and  thfi  S-bstract 
visualization  treatment  group.     These  separate  comparisons 
constituted  a  set  of  contrasts  in  a  two-»by--two  inatri)C  formed  by 
the  absence  or  presence  of  realistic  visualization  \/ersus  the 
absence  or  presence  of  abstract  visualiaations *       This  analysis 
niodel  was  chosen  in  order-  to  facilitate  identification  of  the 
anticipated  additive  effec'C  of  integration  as  an  inteMction  of 
two  factors , 

The  ^primary  analysis  involved  a  five-^faotor  mixed  design 
denoted  RS     in  (A^  @        @         @       )  @  J,    (See  Figure  ID*  Random 
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B  ^  ibitriotion       '    RS^  in  [k^  I    I    1  D^)  f 

Figure  L    Jrisiiry  experiwantal  dggiin. 


subjects  were  nested  in  cells  formed  by  the  four  factors i 
Factor  Ai  Two  levals  of  the  realism  variable , 
1)  no  realism  ands  2)  realistic  visualization. 
Factor  Bi  Two  levels  of  the  abstraction  variable ^ 
1)  no  abstraction  and 5  2)  abstract  visualization. 
Factor  Ct  Tw  levels  of  the  presentation  mode 
variable^  1)  eKternally-paced  and 3   2)  internally-paced. 
Factor  D:   Three  levels  of  the  general  ability  variable^ 
1)  Low 5  2)  Medium  and  3)  High* 
All  subjeets  are  crossed  ^^?ith  four  levels  of  the  dependent 
criterion  variable  C Factor  J)^  1)  drawing  test  score ^   2)  identi- 
fication test  score ^   3)  tBrminology  test  score  and^  U)  the  com- 
prehension test  score.     A  total  criterion  test  score  which  combined 
the  other  H  scores  was  analysed  separately ,     Scores  on  these 
tests  were  transformed  to  standard  scores  before  analysis  of 
results  to  aseura  comparability  of  the  scores . 

because  two  methods  of  iriLfegr^ting  realistic  and  abstract 
visualization  were  being  contrasted 3  the  entire  design  was 
analyzed  twice  with  one  oblong  formed  by  the  A  and  B  factors 
(i.e,,  being  changed    A  total  of  3  0  cells  are  involved 

in  the  study, 

A  separate  analysis  was  used  to  compare  the  two  methods  of 
integrating  abBtract  and  realistic  visualization  across  the  pacing 
and  general  ability  factors.     This  secondary  design  was  denoted 
RS^CA2^   i  Bj'  a  Cg»J  @  J^^     (See  Figure  2). 
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W  (Mode) 


Hybrid  1 
A  •  (Integratiori) 

Both  2 

123 
Low     Med*  High 

C   (General  Ability) 
Figure  2  ^      SeGondary  Experimental  Design 


The  library  prograjn  ANOVR  was  used  for  the  principal 
analysia  of  varlanQe  computations.     This  program  will  analyze 
up  to  four  between  subjects  factors  and  up  to  four  within  subject 
factors  in  a  single  design.     The  library  program  FOLUP  was  used 
for  the  follow-up  comparisons  of  means  after  significant  F 
tests.     The  Tukey  Wholly  Significant  Difference  test  was  speci-* 
fied  in  order  to  control  the  faniily'^wise  risk  of  error  of  falsely 
rejecting  a  true  null  hypothesis  (type  I  error). 
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Results* 

In  the  analysis  of  variance  procedures  for  both  the 
immediate  and  delayed  ppsttests,  the  two-v/ay  interaction  effects 
which  aompared  the  four  types  of  visualization  were  non-sig- 
nificant.    In  the  analyseB  comparing  the  two  inethods  of  inte- 
grating abstract  and  pealistic    visualiEation  the  main  effects 
of  interests  were  similarly  non-significant*     Accordingly ^  the 
analysis  of  both  the  liniiiediate  and  delayed  posttest  scores 
failed  to  reject  the  null  hypotheses.       Follow-up  tests  were 
conducted  in  a  post-hoc  investigation  of  other  significant 
effects. 

Immediate  Posttest  Findings:  Hybrid  Visual  as  the  Method  of 
Integrating  Abstract  and  Realistic  Visualization 

In  the  analysis  of  variance  with  the  four  repeated  measureSs 
when  the  hybrid  visual  wae  the  method  of  integrating  abstract 
and  realistic  visualization ^  the        BC,  BJ  and  CJ  effects 
produced  significant  F  ratios,  ^^^^    In  the  analysis  for  the  total 
criterion  test  scores  only^  the  D  and  BC  effects  were  signifi- 
cant*  Follow-up  tests  were  conducted  on  these  jneans  only. 

As  expected  teste  on  the  main  means  of  D  averaged  over 
all  four  tests,  as  well  as  for  the  total  criterion  test  scores  5 
indicated  that  high  general  ability  students  achieved  significantly 


Tables  1-10  in  Appendix  A  present  the  standard  scores  for 
the  imnedlate  and  delayed  posttests. 

Significance  of  tests  on  means  was  at  the  .05  level,  except 
where  noted  otherwise • 


highar  scores  than  mediuin  and  lov^  ability  students.^    Similarly 5 
mediuni  general  ability  studeiits  achieved  significantly  higher 
scores  than  low  ability  students. 

Tests  on  tha  BC  meanss  a^?eraged  over  all  four  tests  5  as 
well  as  for  the  total  criterion  test  scores ^  indicated  that  when 
abstract  visualization  accompanied  the  instruction 5  students 
receiving    externally-paced  Instruction  achieved  significantly 
higher  scores  than  students  receiving  self -paced  Instruction, 

Tests  of  the  BJ  ineans  yielded  no  signif ioant  diffarences. 

Tests  on  the  CJ  means  indicated  that  students  receiving  self- 
paced  instruction  achiaved  significantly  higher  scores  on  the 
identification  test  than. students  who  received  externally-paced 
instruction.  The  results  suggeated  these  specific  findings f 

1*       When  abstract  visualization  accompanied 

instruction ^  externally-paced  instruction 
was  more  effective  than  self-paced  Instructionj 
for  all  criterion  tests  and  all  levels  of  general 
ability , 

2,       Self -paced  Instruction  was  more  effective 
than  externally--paced  Instruction  for  the 
identification  test  5  regardless  of  the  type 
of  visualization  or  level  of  general  ability* 

Imnediate  Posttest  Findings:     Both  Types  of  Visuals  Used  Together 
as  the  Method  of  Integrating  Abstract  and  Realistic  Visualization 

In  the  analysis  of  variance  i^ith  four  repeated  measures  5  when 
both  visuals  used  together  was  the  method  of  integrating  abstract 
and  realistic  visualisations  the  BC,  and  CJ  effects  produced 

significant  F  ratios.       In  the  analysis  for  the  total  criterion  test 
scores  only 3  the  A,  D,  and  BC  effects  were  significant.  Follow-up 
tests  were  conducted  on  these  TOans  only. 

*  Significance  of  tests  on  means  was  at  the  .05  level  except  where 

noted  otherwise.    -  ,  .  33^ 
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Tests  ori  the  rmin  A  means 5  averaged  over  all  four  tests 5 
as  well  as  fqr  the  total  criterion  test  scoree^  indicated  that 
students  who  received  instruction  accompanied  by  realistic  visua- 
lization achiived  significantly  higher  scores  than  students  who 
received  instruction  ^hich  warn  not  accompanied  by  realistic 
visualization. 

Again  as  expected ^  tests  on  the  ^ain  D  means ^  averaged  over 
all  four  tests 5  aa  well  as  for  the  total  criterion  test  scores ^ 
indicated  that  high  general  ability  students  achieved  significantly 
higher  scores  than  medium  and  lo^  ability  students.     Similarly 3 
mediuin  general  utility  students  achieved  significantly  higher 
scores  than  low  ability  .students. 

Tests  on  the  BC  means  5  averaged  over  all  four  tests ^  as  well 
as  for  the  total  criterion  test  scores^  yielded  no  significant 
differences. 

Tests  on  the  CJ  means  yielded  no  significant  difference, 

Iimnediate  Posttest  Findings:     Comparison  of  the  Two  Methods 
of  Integrating  Abstract  and  Realistic  Visualization 

In  the  analysis  with  four  repeated  measures,  when  the  two 
methods  of  integrating  abstract  and  realistic  visualization  were 
compared,  the        and  B'J  effects  yielded  significant  F  ratios.  In 
the  analysis  for  the  total  criterion  test  only,  the        effect  was 
significant , 

Tests  on  the  nQain  C 'means ^  averaged  over  all  four  tests,  as 
•well  as  foP  the  total  criterion  test  scores,  indicated  that  high 
general  ability  students  achieved  significantly  higher  scores  than 
low  ability  students.     Similarly,  medium  general  ability  students 


achieved  significantly  highar  scores  than  low  ability  studants* 
However^  high  and  mediiiin  general  ability  scopes  wa^e^not  sig- 
nificantly different,  ' 

Tests  on  the  B'J  means  Indicated  that  students  reaeiviiig 
extepnally'-paGed  instruction  achieved  significantly  higher  scores 
on  the  drawing  test  than  students  who  received  self*-pacad 
instruction. 

The  results  suggested  these  speeific  findingsr 

1 .  Low  general  ability  students  achieved  signif ioantly 
lower  scores  than  high  and  medium  ability  studenta^ 
for  all  criterion  measures  5  regardlees  of  the  type 
of  visualization  or  pacing*     However 5  high  and 
medium  ability  did  not  differ  signif iGantly  in 
achievement - 

2,  Externally-Spaced  instruction  was  more  effeetive 
than  s^f-paced  instruction  for  the  drawing  test. 


Immediate  Posttest:  Discussion 

The  findings  of  the  imnediate  posttest  failed  to  reject  the 
null  hypothesis  that  the  various  types  of  visualization  are 
equally  effective : 

'  ^  ^  ^2^1  ^  ^2^2   ^Hybrid)  '  k^B^  (Both) 

More  specif ically 5  the  findings  do  not  support  the  notion  that  the 
integration  of  abstract  and  realistic  visualiEation  will  generally 
improve  the  effectiveness  of  instruction. 

However 5  findings  from  the  analysis  in  which  both  types  of 
visuals  were  used  together  to  integrate  abstract  and  realistic 
visualisation,  that  instruction  accompanied  by  realistic  viaualiza-- 
tion  is  more  effedtive  than  instruction  without  realistic  visualization^ 
is  of  related  interest*     This  effect ^  in  conjunction  with  a  similar 
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effect  of  the  abstraction  variable  (B) ^  was  a  necessary  condition 
for  the  hypothesized  additive  effect  of  eombinlTig  abstract  and 
realistic  visualization* 

This  finding  le]nas  general  support  to  the  ''realisjn  theories" 
(Dale  5  19H6j  Morris ^  194  6 1  Carpenter^  19S3)  which  suggest  that 
as  raalisni  in  pictiires  increases j  so  too  does  their  instructional 
effectiveness.     The  finding  may  also  be  explained  by  the  basic 
research  cited  by  Dember  (I960),  Yarbus  (1967),  Kahneman  (igTS), 
and  Haber  (19  73)  which  suggests  that  complex  visuals  serve  to 
attract  attention  and  increase  exploration* 

The  same  effect  approached  but  did  not  reach  significance 
in  the  analysis  with  the  hybrid  visual  as  the  method  of  inte- 
grating abstract  and  realistic  visualization.     This  failure  to 
achieve  significance  may  have  been  due  to  the  fact  that  the 
hybrid  visual  contains  a  smaller  realistic  coiiiponent  than  the 
complete  color  photo  included  when  both  types  of  visiials  are 
presented  together*       The  benefits  of  realistic  vlsualiEation 
may  be  lost  when  the  complete  realistic  visual  is  not  used* 

The  findings  from  the  analysis  with  the  hybrid  visual  as 
the  method  of  integrating  abstract  and  realistic  visualizatiorij 
that  the  presence  of  abstract  visualization  enhances  eKternally- 
paced  instruction,  is  in  keeping  with  earlier  findings  by  Dwyer 
(1972  5  p*    87)*     One  possible  explaning  for  this  finding  is  that 
in  ei<tern4lly-=paced  instruction  students  are  compelled  to  taKe 
advantage  of  the  edited  cues  in  abstract  visualization  because 
of  the  limited  time  during  which  they  may  interact  with  them* 
Conversely 5  students  receiving  self -paced  instruction  may  not 


be  sufficiently  aroused  to  attend  to  these  abstract  cues  when 
time  is  unlimited. 


tion  was  more  effective  than  externally-pacad  instruction  for 
the  kind  of  learning  measured  by  the  identification  teat  is  not 
surprising i     In  factg  the  same  general  advantage  is  seen  for 
all  tests,  although  the  effect  was  significant  only  for  the 
identification  test.     The  ability  on  the  part  of  the  learner  to 
interact  with  the  instruction  at  his  own  rate  apparently 
enhances  the  kind  of  learning  measured  by  the  identification 
test . 

In  the  analysis  with  only  the  two  methods  of  integrating 
abstract  and  realistic  visualization  ^  externally--paced  instruction 
resulted  in  greater  achievement  than  self-spaced  instruction  for 
the  drawing^  identification 5  and  the  terminology  test*  However^ 
only  the  drawing  test  results  were  significant.  Apparently 
the  integration  of  abstract  and  realistic  visualization  enhances 
externally-paced  instruction  to  a  greater  extent  than  self-paced 
instruction  (except  for  the  comprehension  measure).     This  trend 
lends  limited  support  to  the  researcher's  expectancy  that 
integration  may  facilitate  effectiveness.     One  possible  explanation 
is  that  students  receiving  externally-paced  instruction  were  aided 
in  extracting  relevant  cues  in  a  limited  time  by  the  presence 
of  both  abstract  and  realistic  visualization* 

The  performances  of  hlghj  medium  and  low  general  ability 
students  were  significantly  different  in  the  two  analyses  with 
different  methods  of  integrating  abstract  and  realistic 


The  finding  from  the  same  analysis  that  self-paced  instruc- 
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visualization.     However^  in  the  analysis  comparing  only  the  two 
methods  of  integration,  the  high  and  medium  general  ability 
groups  *  performances  wbtq  not  significantly  different .  This 
finding  suggests  that  integration  of  realism  and  abstraction 
serves  to  reduce  the  difference  in  performance  between  high  and 
medium  general  ability  students.     These  students  may  be  better 
able  to  take  advantage  of  the  integration  of  visualization 
because  they  can  benefit  more  from  the  realistic  component. 

Delayed  Posttest  Findings:     Hybrid  Visual  as  The  Method  of 
Integrating  Abstract  and  Realistic  Visualization 

In  the  analysis  of  variance  with  the  four  repeated  measures ^ 
when  the  hybrid  visual  was  the  method  of  integrating  abstract 
and  realistic  vxsualizatlonj  the  Dj  BCJ^  and  BDJ  effects  produced 
significant  F  ratios.     In  the  analysis  for  the  total  criterion 
test  scores  onlyj  the  D  effect  was  significant.  Follow-up 
tests  were  conducted  on  these  means  only. 

The  D  means  in  the  analysis  with  the  four  repeated  measures 
were  not  of  Interest  because  they  were  Involved  in  the  significant 
BDJ  interaction.     However,  tests  of  the  D  means  for  the  total 
criterion  test  only  Indicated  that  highg  medium,  and  low  general 
ability  students  respectively  each  scored  significantly  higher 
scores  than  the  next  group. 

In  follow-up  tests  on  the  BCJ  interactions  the  BC  means 

r 

were  tested  at  each  level  of  J.     These  tests  yielded  no  signifi- 
cant differences' 

In  follow-up  tests  of  the  BDJ  interactions  the  BD  means 


were  tested  at  each  level  of  J.     There  were  no  oignif leant 
differences  between        and        at  a  given  level  of  general 
ability  (D)  for  any  criterion  teste.       Significant  differences 
in  the  D  means  at  given  levels  of  B  and  J  indicate  that  the 
presence  of  abstract  visualization  tends  to  increase  the 
difference  in  achievement  between  high  and  low  ability  students. 
The  results  suggested  these  specific  findings: 

1.  High  general  ability  students  improved 
moderately^  though  not  significantly  on 

all  criterion  tests  when  abstract  visualization 
accompanied  instruction. 

2.  Medium  general  ability  students  improved  moderately ^ 
though  not  significantly 5  on  the  drawings  Identi- 
ficationj  and  terminology  tests  when  abstract 
visualization  accompanied  instruction.     On  the 
comprehension  test^  a  trivial  opposite  effect 
occurred . 

3.  Low  general  ability  students'  achievement  declined 
^%moderately 5  though  not  significantly ^  on  the 

drawings  identif ication^  and  terminology  tests 
when  abstract  visualization  accompanied  instruction. 
On  the  comprehension  testg  the  opposite  effect 
occurred. • 


Delayed  Posttest  Findings:  Both  Types  of  Visuals  Used  Xogethe 
as  the  Method  of  Integrating  Abstract  and  Realistic  Visualization 


In  the  analysis  of  variance  with  four  repeated  measures ^ 
when  both  visuals  used  together  was  the  method  of  integrating 
abstract  and  realistic  visualization,  the  and  BCJ  effects 

produced  significant  F  ratios.     In  the  analysis  for  the  total 
criterion  test  scores  only^  the  A  and  D  effects  were  significant. 
Follow-up  tests  were  conducted  on  these  means  only. 

Tests  of  the  main  A  means g  averaged  over  all  four  tests 5 
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as  well  as  for  the  total  criterion  test  scores  5  indicated 
that  students  who  received  instruction  accompanied  by  realistic 
visualization  achieved  significantly  higher  scores  than  students 
who  received  instruction  which  was  not  accompanied  by  realistic 
visualization , 

Tests  on  the  D  means ^  averaged  over  all  four  tests ^  as  well 
as  for  the  total  criterion  test  scores 5  indicated  that  high 
general  ability  students  achieved  significantly  higher  scores 
than  Tnediuni  and  low  ability  students*       Similarly ^  medium  general 
ability  students  achieved  significantly  higher  scores  than  low 
ability  students. 

In  follow-up  tests  on  the  BCJ  interactions   the  BC  means 
were  tested  at  each  level  of  J.     The  only  significant  difference 
occurred  with  the  comprehension  test  scores.     These  results 
indicated  that  for  eKternally-paced  instruction 5  students  who 
received  instruction  accompanied  by  abstract  visualization 
achieved  significantly  higher  scores  on  the  comprehension  test 
than  students  who  received  instruction  which  was  not  accompanied 
by  abstract  visualization* 

The  results  suggested  these  specific  findings: 

1,  Students  who  received  instruction  accompanied 

by  realistic  visualization  achieved  significantly 
higher  scores  on  all  criterion  measures  than 
students  who  received  instruction  which  was  not 
accompanied  by  realistic  visualization,  regardless 
of  the  pacing  or  level  of  general  ability, 

2,  Externally-paced  instruction  accompanied  by  abstract 
visualization  is  more  effective  for  the  kind  of 
learning  measured  by  the  comprehension  test  than 
external ly-paced  instruction  which  is  not  accompanied 
by  abstract  visualization . 

er|c  4i: 


399 


Delayed  Posttest  Findings:   Comparison  of  the  Two  Methods  of 
Integrating  Abstract  and  Realistic  ViBualization 

In  the  analysis  of  variance  with  the  four  repeated  measures ^ 
when  the  two  methods  of  integrating  abstract  and  realistic 
visualization  were  compared^  the  C%  B^CJp  and  A^C*J  effects 
yielded  significant  F  ratios^     Only  the  A*C'J  interaction  was  of 
interest  because  the  lower  order  interaction  and  main  effect 
were  involved  in  it  and  uninterpretable <     In  the  analysis  for 
the  total  criterion  test  only^  the  C  and  B-C*   effects  yielded 
significant  ratios.     Accordingly 5  only  the  B'C*  interaction  was 
interpretable . 

In  follow-up  tests  on  the  A'C'J  interactions  the  A-C* 
means  were  tested  at  each  level  of  J*     The  general  trendy  with 
one  exception 5  was  for  the  achievement  of  each  ability  group 
to  improve  moderately 5  and  in  one  case  significantly ^  when 
abstract  and  realistic  visualization  were  integrated  by  presen- 
tation of  both  types  of  visuals  simultaneously  instead  of  by  use 
of  the  hybrid  visual*     Low  ability  students  achieved  significantly 
higher  scores  on  the  drawing  test  when  both  abstract  and 
realistic  visuals  were  presented  simultaneously.     The  one  exception 
to  the  trend  was  found  in  the  performance  of  the  high  ability 
students  on  the  drawing  test*     The  achievement  of  these  students 
was  higher 3  though  not  significantly ,  for  the  hybrid  visual* 

Tests  of  the  A'C  means  between  levels  of  C  at  each  level 
of  A-   and  J  revealed  that  the  presentation  of  both  types  of 
visuals  reduced  the  difference  in  performance  on  the  drawing 
test  between  the  high  and  low  general  ability  groups.     No  similarly 
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str*ong  effect  was  apparent  for  the  other  criterion  tests . 


Tests  on  the  B-C-  me^ns  for  the  total  criterion  test  scores 
revealed  no  significant  differences  between  levels  of  B*  at^  a 
given  level  of  general  ability  CC*).     Similarly^  there  were  no 
significant  differences  between  the  three  general  ability  groups 
at  B^',     However^  at       ^   (the  presence  of  abstract  visualization) 
high  ability  students  saored  significantly  higher  than  jnedium  and 
low  ability  students. 

The  results  sugested  these  specific  findings* 

1.  With  one  except ion ^  all  ability  groups  improved 
moderately  andp  in  some  cases  significantly, 
when  abstract  and  realistic  visualization  were 
integrated  by  presentation  of  both  types  simul- 
taneously instead  of  through  us-a  of  the  hybrid 
visual  * 

2.  The  drawing  test  performance  of  low  ability 
students  was  significantly  improved  by  use  of 

both  types  of  visuals  5   instead  of  the  hybrid  visual* 

3.  Use  of  both  types  of  visuals  reduced  the  difference 
in  performance  on  the  drawing  test  by  high  5  inedium 
and  low  ability  groups* 

f 

4.  For  self-paced  instructionj  high  ability  students 
achieved  significantly  higher  scores  than  medium 
and  low  ability  students* 


Delayed  Posttes t i  Discussion 

As  in  the  iminediate  posttest,  the  findings  of  the  delayed 
posttest  failed  to  reject  the  null  hypothesis  that  the  various 
types  of  visualization  are  equally  effective.     However^  the  findin 
do  indicate  that,  at  least  for  low  general  ability  students' 
performance  on  the  drawing  test^  presentation  of  both  types  of 


visuals  is  more  effective  than  use  of  the  hybrid  visual  as  a 
means  of  integrating  abstract  and  realistic  visualization.  This 
effect  was  also  apparent ^  though  not  significant  for  other 
ability  groups  performing  on  other  tests,       Relatedly,  both 
types  presented  together  also  reduced  the  difference  in  per- 
formance on  the  drawing  test  of  high  and  low  ability  groups. 
These  findings  may  possibly  be  eKplained  by  the  increased  amount 
of  realistic  visualization  available  when  both  types  of  visuals 
are  presented  simultaneously.     Also,  the  findings  are  supportive 
of  Hudgins   (1377)  who  suggests  possible  advantages  to  a  multi--stage 
use  of  abstract  and  realistic  visualization. 

The  finding  in  the  analysis  in  which  both  types  of  visuals 
were  used  together  to  integrate  abstract  and  realistic  visualization 
is  more  effective  than  instruction  without  realistic  visualization 
is  also  of  interest.  This  finding  lends  general  support  to  the 
"realism  theories"   (Daleg   1946;  Morris^   1946;  Carpenter,  1953) 
which  suggest  that  as  realism  in  pictures  increasesj   so  too  does 
their  instructional  effectiveness.     The  findings  may  also  be 
explained  by  the  basic  resaarch  cited  by  Dember  (1960) ,  Yarbus 
(1967),  Kahneman  (1973),  and  Haber  (1973)  which  suggests  that 
complex  visuals  serve  to  attract  attention  and  increase  exploration • 
This  effect  was  not  present  in  the  design  in  which  abstract  and 
realistic  visualization  vjere  integrated  by  means  of  the  hybrid 
visual*     One  possible  explanation  for  this  may  be  that  the  hybrid 
visual  contains  less  realistic  area  than  the  total  color  photograph. 

The  effect  of  abstract  visualization  was  inconsistent.  In 
the  analysis  with  the  hybrid  visual  as  the  method  of  integrating 
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abstract  and  realistic  visualization,  high  and  medium  general 
ability  studente  benefited  moderately ^  though  not  signif icantly 
on  most  tests  from  the  presence  of  abstract  visualization. 
Howevers  low  ability  students-     scores  declined  moderately  though 
not  signif icantly 3  on  most  tests  when  abstract  visualization 
accompanied  instruction*     Some  kinds  of  learning  by  low  ability 
students  may  be  susceptible  to  long  term  forgetting  because  they 
were  not  sufficiently  aroused  by  abstract  visualization. 

In  the  analysis  in  which  both  types  of  visuals  were  presented 
simultaneously  to  integrate  abstract  and  realistic  visualization, 
abstract  visualization  significantly  enhanced  the  effectiveness 
of  externally-paced  instruction  as  measured  by  the  comprehension 
test*     This  finding  is  consistent  with  earlier  findings  by 
Dwyer  (1972).       Apparently 5  the  abstract  visualization  enabled 
students  to  extract  relevant  cues  which  improved  learning. 


The  results  of  both  posttests  suggest  that  designers  of 
visualized  intruction  may  expect  to  enhance  its  effectiveness 
under  general  conditions  by  inclusion  of  realistic  visualization. 
Inclusion  of  abstract  visualization  cannot  be  expected  to  improve 
effectiveness  under  general  conditions  and  should  be  based  on 
consideration  of  pacings  general  ability  and  type  of  instructional 
obj  ect ive . 

Although  the  integration  of  abstract  and  realistic  visualization 

m 

does  not  appear  to  generally  improve  effectiveness^  it  may  serve 

to  enhance  externally  paced  instruction  and  may  reduce  the  difference 
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in  achievement  between  students  of  different  general  ability 5 
particularly  when  both  typeB  of  visualization  are  presented 
together. 
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6,7*       8,53  7.08' 

*A  ^ 

(line  Drawing)  s 

M,ll      55.22  57.89 
7,7?       8.86  2.76 

^3,6?      M.U  W.78 
e.15  8,50 

(Real 
Photop^apli)  e 

M.67      1^8.11  51.33 
10,98       8,04  10.92 

43.56      51, ?a    58. 78 

9.89    8.53  m 

(Hybrid)  s 

^3.67      59.56  58.89 
5.55      12.00  10,98 

lii.67      khkk  58.22 
3i5*       8.95  10.02 

A^B^  ! 
(loth)  s 

52.22      56.33  53.5^ 
lli^l       7.86  12,88 

t7.W     51'78  53'55 
8.57      7.82  6.89 

TABM  2 

iraVlEDIATE  POSTTEST  STANDARD  SCORES,  IDlNTIPlCAriON  a?EST 


Externally-Paced 
General  Ability  Groups 

Self -Paced 
General  Ability  Groups 

Low          Medium  High 

Low          Medium  Hi^ 

(Control)  B 

4-2,89          ii-8.22  51.89 
5.3              9.6  13.2 

^1-8,78          43.56  52,67 
5,4             6.4         10.3  . 

hh  ^ 

(Line  Drawing)  s 

^^8.56          54.56  58.00 
5.0           10,0  10.7 

43,89          47.00  47,22 
6.0            10.9  10,7 

^2^1  ^ 
(Real 
Photograph)  s 

^9.56          50,11  5^,56 
10,0            11. if  8.3 

43,22          50.78  57.56 
7.8              8,5  5.2 

AgBg  X 
(Hybrid )  s 

k6,56          56. W  62,11 
7.5            11.6         10, J 

43.22          45.22  60.33 
3.0              9.0  12.1 

AgBg  X 
(Both)  s 

49.89          57.11  ^+9^78 
8.6           '8.7  12.3 

45.33         53«22  55.44 
9.4           10.7  8.6 

n^9 
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TABLE  3 

IiyMEDIATl  POSTTEST  STANDARD  SCORES,  TERMINOLOGY  TEST 


m 

Ixternally-Paoed 
Ganeral  Ability  Groups 

Self -Paced 
Oeneral  Ability  Groups 

Low 

Medium 

Hiih 

Low          Mediiim  Hl^ 

-1^1 

X 

45.33 

51.67 

48.22 

48.  J6 

49.11 

56.78 

(Control) 

s 

6.4 

7.3 

11.3 

8.7 

9.9 

8.6 

1  2 

X 

48.56 

50.44 

59.22 

43.78 

47.00 

49.11 

(liine  Drawing) 

s 

6 .4 

/.o 

l4. 8 

D  a  U 

11.9 

7.0 

(Real 
Hiotograph) 

X 
s 

44.00 
7.4 

52.89 
9.7 

51.00 
12.4 

47.22 
5.3 

50.78 
10,0 

60,67 
7.0 

X 

44.56 

55.44 

60,11 

41,78 

50.11 

58.67 

(Hybrid ) 

s 

7.8 

10.5 

16.1 

5.04 

10,9 

10.8 

AgBg 

X 

46.33 

52.78 

52.89 

44.11 

52.44 

58.56 

(Both 

s 

5.8 

10.7 

10,4 

4.7 

8.3 

12.1 

n-9 
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TABIE  I* 

IMMEDIATI  POSTTIST  STANDARD  SCORIS ,  COMPREHINSION,  TEST 


Ixternaily-Paced 
General  Ability  Groups 

Self -Paced 
General  Ability  Groups 

Low 

Medium 

High 

Low 

Mediuin 

Hi^ 

1  1 

X 

51.67 

50.78 

47.22 

51.56 

53.78 

(Control) 

s 

8,8 

10.8 

8.5 

6.3 

12 , 0 

X 

50.22 

50.22 

52.33 

46.78 

47.89 

50.00 

(Line  Drawing' 

s 

6.7 

10.9 

14,2 

6.6 

8,0 

5.3 

X 

^5.78 

49,22 

56.22 

50,22 

49.44 

58.66 

(Real 
Photograph) 

s 

9.1 

9.7 

9.9 

,   ^  9.2 

8,4 

15.3 

AgBg 

X 

50.78 

54.56 

42,67 

49.78 

57.78 

(Hybrid) 

s 

4.7 

10,6 

■    16 . 3 

5.4 

8.4 

12.4 

X 

45.89 

51.00 

53.44 

46,44 

51.44 

59.89 

(Both) 

s 

• 

8.4 

7.4 

8,6 

6.1 

12,3 

9.8 

na9 
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TABIB  5 

IMIVEDIATE  POSTTEST  STANDARD  SOORIS,  .TOTAL  ORITERION  TEST 


Ix- 

bernally-Paoed 

Self -Paced 

* 

Genaral  Ability  Groups 

General  Ability  Oroups 

Low 

Medium 

Hi^ 

Low 

Medl\M 

Hi^ 

X 

kz,zz 

50,11 

51.00 

46.44 

46,44 

52,89 

( nnri+r*nl  \ 

4.3 

7.9 

11.8 

6.4 

9.4 

X 

48.78 

53.44 

58.56 

43 . 22 

47,00 

48,44 

(Line  Drawing) 

s 

3.3 

8.7 

11.7 

5.9 

10,0 

8.6 

X 

46.78 

49.89 

53.89 

44,89 

50,78 

60,78 

(Real 

Photograph ) 

s 

10,3 

9.2 

10.9 

7.3 

9.2 

9.1 

X 

43.78 

57.22 

61,00 

40,67 

48.33 

60,67 

(Hybrid ) 

s 

5.3 

11.1 

14.8 

2.3 

9.3 

12,2 

X 

48,78 

55.56 

53.11 

45.11 

52.67 

57.89 

(Both) 

s 

6.5 

9.0 

12.0 

5.4 

9.1 

5.3 

n^9 
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TABIE  6 

DELAYID  POSTTEST  STANDARD  SCORES,  DRAWING  TEST 


Externally-Paced 

Self "Paced 

General  Ability  Groups 

General  Ability  Groups 

Low 

Medium 

High 

Low 

Medium 

High 

X 

^1-2,89 

51.11 

50.22 

46.11 

45.78 

54,22 

(Control) 

s 

3.59 

10.90 

10.35 

7.15 

6.53 

7,69 

A,  B« 

ks.  89 

51.89 

41.56 

49.11 

53.44 

(Line  Drawing) 

s 

Q  nh. 

t  *  {  ( 

y  t  w  W 

&  f  00 

10. 41 

11,35 

X 

50.11 

49.89 

51.67 

45.89 

52.11 

57.  U 

(Real 

Photograph) 

s 

8.38 

7.29 

9.38 

9.55 

9.49 

13,06 

X 

40.89 

53.22 

58.89 

41.22 

50.78 

59 ,44 

(Hybrid) 

s 

2.15 

9.85 

13.47 

2.86 

10.35 

13.05 

AgBg 

X 

51.22 

53.33 

53.78 

47.11 

51.67 

55.22 

(Both) 

B 

10.  78 

8,17 

12,36 

6.58 

7.40 

14.21 
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TABLE  7 

DEMYED  POSTTEST  STANDARD  SCORES,  IDENTIPICATION  TEST 


Externally-Paoed 
General  Ability  Groups 

Self "Paced 
General  Ability  Groups 

Low          Medium  High 

Low 

Medium 

Hi^ 

A  IS 

X 

49.78 

48.67 

48,11 

45.56 

52.22 

(Control) 

s 

5.92 

16.84 

13.76 

8.80 

5.50 

8,18  . 

12 

X 

50.67 

58.67 

47.33 

45.56 

56.89 

(Line  Drawing) 

S 

3.81 

7.52 

12.84 

5. 81 

6,46 

8.59 

(Real 

X 

52.78 

49.44 

53.78 

45.56 

50.33 

55»11 

Photo^aph) 

s 

12,71 

5.27 

12,01 

8.71 

10,61 

13.49 

X 

^5.33 

51.11 

55.22 

45.89 

51.00 

58.78 

(Hybrid) 

s 

3.28 

8,91 

14.11 

8.55 

10,05 

11.33 

X 

48,22 

52.33 

53.00 

45.33 

51.78 

61.67 

(Both) 

5 

5.93 

8.63 

12.01 

6,86 

7.89 

13.45 

n^9 
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TABI£  8 

DELAYS  POSTTIST  STANDARD  SCORES,  TERMINOLOGY  TEST 


* 

Ext  er nally-Pac  e  d 
General  Ability  Groups 

Self -Paced 
General  Ability  Groups 

Low 

Medium 

Hl^ 

Low 

Medium 

X 

53.33 

50.ifif 

ifif.  11 

if7.00 

50.56 

(Control) 

B 

5.88 

11, ii.7 

7.09 

8.36 

8.96 

12.51 . 

A 

^5.67 

55.89 

51.67 

5^.67 

g 

no 

f.Uf 

^  on 
0  i  0& 

(Real 
Photogra^) 

X 

s 

i|-6.00 
5.29 

if  8. 33 
9.19 

58.56 
16.79 

if  8. 11 
9.89 

52.78 
9.56 

58.00 

10.81 

X 

i4.if.78 

50.89 

ij.9.00 

if5.89 

if5.67 

61.56 

(Hybrid) 

s 

6.22 

8,24 

8.05 

if.if3 

6,95 

ia,6i 

1+6.56 

51.00 

52.89 

W,33 

if7,ifif 

67,22 

(Both) 

s 

8.32 

9.03 

6.85 

7.0if 

7.88 

11,2? 

1 

n^9 
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TABIM  g 

DELAYED  P0STT1S5  STANDARD  SCORES,  COMPRHilNSION  TEST 


Ext ernally-Pac  ed 
General  Aoility  Groups 

Self -Paced 
General  Ability  Groups 

Low          Mediiain  Hi^ 

Low 

MediiTO 

X 

^1.78 

45.67 

50.78 

44,33 

49,89 

51.78 

(Control) 

s 

5.68 

8.57 

6,20 

11,19 

9.19  . 

X 

51.56 

49.56 

56.78 

50,00 

47.33 

50.56 

(Line  Drawing) 

s 

10.  08 

4.93. 

17.69 

4.04 

16 ,19 

B.97 

X 

46.89 

51.56 

45.44 

53.67 

55.56 

(Real 
Photo^aph) 

s 

8.3^ 

7.87 

10.97 

9.22 

10.46 

16,00 

AgBg 

X 

47.00 

48.89 

52.33 

46.44 

^7.67 

(Hybrid ) 

5 

7.40 

10.08 

7.19 

7.35 

10.19 

10.36 

AgBg 

X 

51.56 

51.56 

51.77 

46,67 

50.67 

66.33 

(Both) 

s 

7.94 

13.5^ 

8,50 

10,04 

11.64 

12.48 

n^9 
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TABm  10 

DEIAYED  POSTTEST  STANDARD  SCORIS ,  TOTAL  ORITERION  TEST 


m 

Ext  er nally-Pac  a  d 
General  Ability  Groups 

Self -Paced 
General  Ability  Groups 

Low 

Medium 

Higji 

Low 

Medium 

X 

50.11 

50.00 

^5«11 

^1-6. 00 

53,00 

(Control) 

s 

3.50 

I2.I1-O 

10.25 

7,06 

6.87 

8,03 

X 

k7,oo 

49.6? 

58.89 

ks.oo 

55,11 

r-»^  J^^\ 

s 

C     1  /' 

5.77 

9.75 

X 

52.56 

57.78 

(Real 
Photo^aph) 

s 

9.70 

12.88 

9,^9 

10.27 

13,20 

X 

^2.89 

55.56 

^3.33 

^4-8.89 

61,33 

(Hybrid) 

s 

1.96 

2.92 

12.65 

^.39 

7.75 

12,60 

AgBg 

X 

^^9.22 

52.89 

53.4^ 

^5.33 

50.78 

6k, 

(Both) 

s 

7.61 

8.85 

9.30 

6,0k 

9.02 

14.15 

n=9 
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Efficiency  of  Students'  Aohievement  Using 
Black/V/hite  and  Color  Coded  Learning  and  Test  Materials 

Abstract 

The  purpose  of  this  study  v/as  to  compute  and  analyze 
descriptiye  indices  of  efficiency  assessing  the  relative 
efrectiveness  of  a  verbal  ajid  visual  color  code  applied  to 
self-paoed  learning  and  testing  materials,    The  sample 
consisted  of  175  college  subjects  randomly  assigned  to  one 
of  four  treatment  conditions  receiving  hlaok/white  (B)  or 
color  coded  (C)  materialss  B  learning  and  testing  materials, 
B  learnins  materials  but  0  testing  materials^  C  learning  and 
testing  materials^  or  0  learning  materials  but  B  testins 
materials* 

Materials  consisted  of  a  21  page  learning  booklet  on 
the  human  heart  and  an  80  item  test  measuring  different 
levels  of  cognitive  knov/ledgei  content  v/as  identical  except 
that  in  some  treatment  conditions  a  color  oode  having 
raeaningful  physical  form  and  assooiative  value  v/as  applied* 
Subjects  read  through  the  learning  booKlet  follov;ed  immedi-^ 
ately  by  a  test;  six  weeks  later  the  identical  test  was 
administerGd  to  assess  delayed  retention.     Time  on  learning 
and  time  on  tests  was  recorded. 

Calculation  of  three  related  efficiency  formulas 
(achievement  per  unit  of  time)  provided  the  descriptive  data 
for  dlsGUSsion,    Data  indicated  that  v/hile  color  coded 
learning  materials  facilitated  achievement ^  additional  time 
for  acquisition  and  for  retrieval  was  necessary.  This 
primary  relationship,  in  addition  to  secondary  relationships , 
v;as  further  discussed  in  light  of  each  efficiency  derivation. 


Efficiency  of  St-udents  •  Ichievenent  Using 
Black/White  and  Color  Coded  Learning  and  Test  Materials 

Purpose  and  Rationale 

Research  during  the  instructional  media  era  of  the 
1950 's  and  1960 's  was  comparative  in  nature  (Torkelson,  1978) 
and  sought  to  identify  device  usefulness  rather  than  to 
identify  the  parameters  impacting  on  elements  of  the  message 
(Dwyer,  1972|  1978).    Questtons  posed  were  to  Justify  using 
media  rather  than  to  optiniize  learning.    It  was  this  later 
need  to  investigate  those  message  design  elements  v;hlch  in 
combination  or  limitation  may  facilitate  optimal  (effective 
and  efficient)  learning  that  this  study  sought  to  supply 
additional  evidence  for.    Specifically  this  investigation 
sought  to  clarify  the  cognitive  value  of  color  when  used  as 
a  coding  device  in  the  design  of  self-paced  learning  and 
testing  materials. 

In  surveying  the  color  literature  researchers 
(laniberski,  1972;  1979i  Berry,  I97i«-!  Chute,  1978)  have  con- 
cluded that  the  significance  of  color  as  a  design  variable  has 
typically  eluded  researchers  who  have  attempted  to  define 
its  u_nlque  role  rather  than  its  possible  interrelated  role 
in  the  learning  process.     In  many  studies  the  use  of  color 
was  often  superseded  by  other  variables  or  not  properly 
isolated  by  ejcperimental  design  to  assess  its  effectiveness. 

Exploratory  investigations  in  cognitive  learning  have 
fo\ind  that  color  may  facilitate  or  inhibit  learning  when  used 
as  an  instructional  cue  or  code  in  message  design  (Kauffman 
and  Dwyer,  197^;  Lamberski,  19755  Berry,  1975|  1976).  While 
there  is  support  for  the  affective  preference  for  color 
and  for  the  physiological  and  developniental  influence  of 
color,  it  is  evident  that  the  cognitive  value  of  color  to  the 
learning  process  must  be  clarified  further  CDwyer,  1978; 
Lamberski,  1979).    Thus,  the  purpose  of  this  study  was  to 
compute  and  analyze  descriptive  indices  of  efficiency 


assessiiig  the  relative  effectiveness  of  a  verbal  and  visual 
ooloT  code  applied  to  self-paced  learning  and  testing  materials* 
It  v/as  hypothesised  that  when  both  learning  and  testing 
materials  utilized  a  color  code^  learning  v/ould  be  most 
efficient  (greatest  performance  or  achievement  per  unit  of 
time)p 

Method 

Sample 

The  sample  consisted  of  I76  (39  male  and  I37  female) 
student  volunteers  from  an  instructional  media  course  at 
The  Pennsylvania  State  University. 

Instructional  Materials 

The  learning  material  (D^^er  and  Lamberski^  1977)  was 
a  revision  of  a  2^000  v/ord  prose  text,  with  accompanying 
simple  line  drav/ing  visualization  and  verbal  labels^ 
oriBinally  published  by  Dwyer  (1972)*    The  content  of  the 
material  consisted  of  the  parts  and  internal  processes  of 
the  human  heart*    Tv/o  versions  of  the  21  page  learning  booklet 
were  developed;  one  was  entirely  blaok/white  (B)  v/hile  the 
other  one  had  a  color  code  (C)  applied. 

The  color  code  v/as  applied  only  to  relevant  (central) 
concepts  or  functions  being  presented  on  a  particular  page 
with  no  color  coding  v/hen  the  concept  or  function  v/as 
redundant  (peripheral)  to  other  instruction  being  presented^ 
The  color  oodo  application  in  the  booklet  was  for  buth  the 
verbal  instruction  (the  ink  color  of  the  printed  concept 
or  function  v/ord)  and  accompanyins  visual  illustration 
(the  identical  ink  color  of  the  printed  concept  or  function 
label,  arrow J  and  shaded  area).    The  six  colors  and  black 
formed  categories  which  related  similar  concepts  and 
functions  or  discriminated  between  the  other  concept  or 
function  words*    Thus,  the  color  code  had  both  a  physical  form 
(visual  with  cues)  and  an  associative  value  (verbal  with 
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meaning) ♦    Such  a  color  code  application  has  been  applied 
in  various  primary  reading  investigations  (Hinds  and  Dodds^ 
1968)^  and  more  recently  color  visualisation  has  been  applied 
and  found  to  have  an  associative  value  (V/inn|  1975;  1977) • 

The  test  material  (Dwyer  and  Lamberski^  1977)  v/as  a 
revision  of  the  battery  of  tests  originally  published  by 
Dv;yer  (1972)*     The  materials  consist  of  four  20-item  subtests 
(summed  together  equal  80  Items)  assessing  hierarohical 
cognitive  knov/ledge  of  drav/ing,  identlf icationy  terminology , 
and  oomprehension.    Two  forms  of  the  test  v/ere  prepared ^  one 
to  follov;  each  of  the  black/v/hite  (B)  and  color  coded  (0) 
learning  material  formats. 

All  materials  were  piloted  prior  to  the  present 
investigation*    Test  instrument  reliability  coefficients 
for  data  in  this  study ^  determined  by  the  Kuder-Richardson 
formula  20  as  being  betv/een  .7^-. 88  for  the  four  respective 
subtests  (Lamberski^  1979) ?  compared  quite  favorably  v;ith 
coefficients  found  in  investigations  using  Dv;yer's  original 
materials  (Dv/yer^  1972). 

Procedure 

Participants  v;ere  randomly  assigned  to  one  of  four 
treatment  conditions;  each  treatment  represented  a  crossing 
of  the  two  levels  -  black/white  (B)  or  color  coded  (0)  * 
for  each  of  the  independent  variables  -  learning  and 
testing  materials^    The  resulting  cells  in  the  design 
therefore  werei  B  learning  and  B  test  materials,  B  learning 
but  C  test  materials,  C  learning  and  0  test  materials,  and 
0  learning  but  B  test  materials.    Subjects  read  the  learning 
booklet  immediately  after  which  they  v;ere  administered  a 
test;  six  weeks  later  the  identical  test  v/as  administered  to 
assess  delayed  retention.    Time  on  the  learning  task  and  time 
on  both  retention  tests  v;ere  recorded  by  each  subject. 
There  were  N  ^  ^  sub-jecto  in  each  treatment  condition. 


Resjults  and  Disousslon 

Three  desoriptive  indices  of  instructional  efficiency 
were  computed  whereby  the  num'ber  of  correct  items  on  the 
immediate  or  delayed  tests  were  divided  by  a  corresponding 
time  measure.    Thus,  the  dependent  variables  for  subgeots 
in  each  treatment  condition  werei  the  mean  achievement 
score  (Table  1)  which  represGnts  the  number  of  correct  items 
(80  items  total)  on  each  respective  retention  test;  the 
mean  learning  time  (Table  2)  vjhich  represents  how  long  it 
took  to  complete  the  learning  materials  (expressed  in 
minutes)  I  and  the  mean  test  time  (Table  j)  v/hich  represents 
how  long  it  took  to  complete  the  testing  materials  (expressed 
in  minutes)  on  each  respectiv©  retention  test.     Thus,  the 
derived  mean  efficiency  values  for  the  three  descriptive 
indices  (Tables  4,5^  &  6)  represent  mean  number  correct  per 
minute  on  each  respective  retention  test.    Data  are  visually 
summarized  in  Figures  1,  2^  and  5*  • 

Calculation  of  the  three  related  efficiency  formulas 
(achievement  par  unit  of  time)  provided  evidence  which 
indicated  that  v/hile  color  coded  learning  materials  facili- 
tated student  achievement,  additional  time  for  acquisition 
and  for  retrieval  v/as  necessary.    Data  showed  higher 
efficiencies  for  immediate  test  results-,  this  simply  reflects 
that  recall  v/as  poorer  at  delayed  retention  teatins.  Also, 
performance  or  achievement  was  more  efficient  when  black/white 
tests  were  used.    Data  did  not  support  the  hypothesis  that 
the  most  efficient  instructional  strategy    v/as  to  pair 
color  coded  learning  materials  with  color  coded  tests.  While 
the  second  efficiency  index  (Figure  2,  Table  5)  using  the 
test  time  measure  was  computed  primarily  for  interest,  the 
first  (Figure  1,  Table  4)  and  the  third  (Figure  3,  Table  6) 
efficiency  index  values  should  be  viewed  as  more  realistic 
since  learning  time  or  the  combination  of  learning  and 
testing  time  will  normally  far  outweigh  testing  time  alone. 
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The  findinss  might  mean  that  color  coding  facilitBted 
the  quality  and  the  quantity  of  the  encoding  arid  the  decodliis 
processes  but  hecause  the  discrimination  amongst  numeroiis 
colors  v;as  a  more  time  coiisumiriq  process^  the  Increasad 
encoding  and  decoding  time  lowers  the  efficiency  index* 
Hov;ever,  the  data  did  s^igsest  that  aolor  coding  of  learning 
materials  could  be  an  efficient  instructional  strategy 
when  coupled  v;ith  the  sirnpler  hlack/white  test*  This 
might  be  an  artiract  of  the  current  data  though^  since 
testing  time  was  slmllax  to  learning  time.    Certainly^  a 
hotter  situation  v;ould  he  to  extend  learning  time  constder- 
ahly  (perhaps  10  folci  nx  more)  vjhile  holdiris  teBtine  time 
constant*     That  should  provide  a  more  realistic  test  of  the 
original  efficiency  hypotliesis. 
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TA3m  -1  - 


Mean  Achievement _Scores 


Treatment 


Retention 

B/B 

B/0 

0/a 

C/B 

Immediate 

52.64 

_  51.34 

61.38 

58.77 

55.03 

Delayed 

39.91 

39.28 

44,20 

43.48 

41.72 

46.28 

^5.31 

52.79 

51.13 

48.88 

TABLE  g  -  Mean  Learning  Time 


B/B 


14,11 


I'reatment 
B/C  0/0 


20.18 


C/B 


16.50 


16.43 


TABLi:  ^  -  Mean  Test  Time 


Treatment 


Retention 

B/B 

B/C 

C/C 

C/B 

Immediate 

__25.49 

26.12 

29.12 

25.89 

26.65 

Delayed 

22.89 

23.08 

^28,25 

24.24 

24.62 

24,18 

24.60 

28.69 

25.07 

25-64 

Table  Notes: 

B/B  =  Black/v;hite  learning  and  test  materials 

B/C  =  Black/white  learning  materials  but  color  test  materials 

C/C  =  Color  learning  materials  and  test  materials 

C/B  ^  Color  learning  materials  but  black/white  test  materials 

Tables  2  and  3t  Values  represent  minutes. 


ERIC 


425 


Immediate  Delayed 


TABLE  4  ^  Efficiency^ 


Retention 

Immediate 
Delayed 


Achievement/Learnin.^  Time 
Treatment 


B/D 

B/C 

0/0 

C/B 

3.??1 

3.^59 

B,0/!=2 

_  3.362 

2,028 

2.631 

2.190 

2.655 

2,571 

3.280 

3.035 

2.616 

3.099 

3.007 

Note.-  Values  represent  mean  number  correct  per  minute. 
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Figure  2  Efficiency.  Achievement/Test  Time 

2,5- 


Imtnediate  Delayed 


TABLE  5  -  Efficiency;  Timff 

Treatment 


Retentton 

B/B 

B/C 

C/C 

C/B 

Immediate 

2.067 

1.966 

a. 108 

2.270 

2.105 

Delayed 

1 . 744 

1.702 

1.565 

1 , 794 

_  1 .7pl_ 

1.906 

1.834 

1.837 

2.032 

1.902 

Note:  Values  represent  mean  number  correct  per  minute. 
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iurBB  Efficiency;  Achievemen|^Learning(iTesf Tim 


Immadiata  Dalayed 


JABLE  6^  Ef^lciencyr  Achlevement/Learnlnf^      T^pt  ^iinfi 

Treatment 


Retention 

B/B 

B/0 

C/C 

C/B 

Immediate 

1.251 

1 , 245 

1.586 

1 . 30^ 

Delayed 

1.079 

1*033 

0,915 

1.067 

1 . 025 

1.205 

1.142 

1.079 

1.227 

1.165 

Notei  Values  represent  mean  number  correQt  per  minute. 
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Mann,  Raymond  E,,  The  Effect  of  Nusjc  and  Sound  Effects 
ojl  the^  Li  s  teni  ng  Comprehens  i  on  of  Fourth  Grade  Studen  ts  > 
North  Texas  State  University,  Marchp  1979,  27pp,t  7  tables, 
bi  b 1 1 ography , 

The  purpose  of  this  study  was  to  determfne  If  the 
addition  of  music  and  sound  effects  to  recorded  stories 
increased  the  comprehension  and  retention  of  Information 
presented  on  tape  for  fourth  grade  students.     Two  versions 
of  four  narrated  stories  were  recorded---one  version  witn 
musfc  and  sound  effectSs  the  other  included  the  narration 
only,    A  listening  comprehension  test  was  administered  after 
listening  to  each  of  the  stories  and  followed  by  a  retention 
test  two  weeks  later.    Students  listening  to  the  music  and 
sound  effects  versions  scored  significantly  higher  on  both 
the  listening  comprehension  and  retention  tests. 


INTRODUCTION 

Listening  1s  one  of  the  first  activities  In  the  life 
of  every  Individual.     Research  has  indicated  that  newborn 
infants  can  make  ''fine  discriminations  in  sounds.  Sounds 
have  meaning  for  babiesi  even  before  they  have  learned 
language''  ( 16 )  , 

Children  learn  to  speak  primarily  through  listening 
to  others.     However^  primary  children  have  difficulty  in 
listening  effectively  to  dialogues  encountered  in  story- 
telling situations  or  in  following  instructions  {3^  4p 
17,).    Many  of  the  reasons  why  children  have  difficulty 
in  listening  have  not  been  identified. 

The  Importance  of  listening  as  a  distinct  learning 
process  has  been  stressed  by  psychologists  and  educators 
for  some  time.     Duker  states  that  listening  *'has  always 
been  and  continues  to  be  the  most  widely  used  human  means 
of  receiving  information*'  (7)  from  others.     An  individual' 
listening  activities  are  basic  In  order  to  learn  to  "speak 
readg  and  wri  te"  (6 ) . 

Studies  by  Rankin  and  Wilt  indicate  children  spend 
over  fifty  per  cent  of  their  school  day  listening  (9,  19) 


and  students  who  score  high  on  listening  comp f ehens i on  tests 
normally  do  better  in  school   (15),     Studies  have  also  shown 

if 

that  students  can  be  taught  to  listen  more  effectively  (11). 

Even  though  listening  has  received  much  recognition 
and  the  importance  of  listening  has  been  stressed  by  many 
educators^  very  little  research  has  been  done  on  what 
factors  contribute  to  one's  ability  and  willingness  to 
listen.     Although  numerous  articles  about  listening  have 
been  written,  few  are  based  on  systematic  research  on 
listening  as  an  information  gathering  psychological  process* 

Also  important  is  the  lack  of  research  on  the  numerous 

instructional  presentation  modeSs  all  of  which  have  an 

effect  on  the  listening  process. 

In  view  of  the  increasing  instructional  use 
of  a  wide  range  of  teaching  materials  and  presen- 
tation media--such  as  video  cassettes,  audiovisual 
teaching  kits,  audio  cassettes^  etc. --the  necessity 
for  such  systematic  investigation  is  readily 
apparent  particularly  In  that  some  me^h'a  may  be 
more  or  less  effective  than  others  (Zw,  p.  32). 

The  increase  of  media  utilization  in  American  schools 

during  the  past  twenty-five  years  has  been  unprecedented 

and  the  potential  educational  significance  of  these  materials 

is  tremendous*     However,  the  instructional  potential  of 

media  can  never  be  achieved  without  a  solid  foundation  of 

research  and  unders  tandi  ng  of  chi 1 dren ' s  1 1 s  teni  ng 

competence  (20) , 
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statement  of  the  Problem 
The  probleni  of  this  study  was  to  deternilne  the  effect, 
if  any,  of  music  and  other  sound  effects  on  listening 
coinprehensi  on  and  ricall  of  fourth  grade  students. 

PurpDses  of  the  Stud^ 
The  purposes  of  this  study  were: 

1.  To  determine  if  fourth  grade  students'  comprehension 
of  stories  recorded  with  music  and  sound  effects  was  greater 
than  straight  narrative  stories* 

2,  To  determine  If  fourth  grade  students'  recall  of 
information  from  stories  recorded  with  music  and  sound 
effects  was  greater  than  straight  narrative  stories* 

3*    To  determine  If  fourth  grade  students'  reading 
ability  was  a  factor  in  comprehension  and  recall  of  Infor- 
mation froni  stories  recorded  with  music  and  sound  effects 
or  those  stories  recorded  without  music  and  sound  effects* 

Hypotheses 

The  following  hypotheses  were  tested  by  statistical 
analysis  of  the  data  collected: 

1*     There  will  be  no  significant  difference  in  the 
comprehension  of  information  by  fourth  grade  students  who 
listened  to  narrated  stories  with  music  and  other  sound 
effects  and  by  those  who  listened  to  the  same  stories 
without  music  and  other  sound  effects, 
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2,  There  will  be  no  significant  difference  in  the 
comprehension  of  information  by  fourth  grade  students  with 
low  reading  abilities  who  listened  to  narrated  stories 
with  music  and  other  sound  effects  and  by  those  who 
listened  to  the  stories  without  music  and  other  sound 
effects  * 

3,  There  will  be  no  significant  difference  in  the 
comprehension  of  information  by  fourth  grade  students  with 
average  of  mediuni  reading  abilities  who  listened  to 
narrated  stories  with  music  and  other  sound  effects  and  by 
those  who  listened  +.0  the  stories  without  inusic  and  other 
sound  effects* 

4,  There  will  be  no  significant  difference  In  the 
comprehension  of  information  by  fourth  grade  students  with 
high  reading  abilities  who  listened  to  narrated  stories 
with  music  and  other  sound  effects  and  by  those  who  listened 
to  the  stories  without  music  and  other  sound  effects. 

5,  There  will  be  no  significant  difference  in  the 
retention  of  information  by  fourth  grade  students  who  listened 
to  narrated  stories  with  music  and  other  sound  effects  and 

by  those  who  listened  to  the  stories  without  music  and  other 
sound  effects. 

6,  There  will  be  no  significant  difference  in  the 
retention  of  Information  by  fourth  grade  students  with  low 
reading  abilities  who  listened  to  narrated  stories  with 


music  and  other  sound  effects  and  by  those  who  listened 
to  the  stories  without  music  and  other  sound  effects. 

7.  There  will  be  no  significant  difference  In  the 
retention  of  Information  by  fourth  grade  students  with 
average  reading  abilities  who  listened  to  narrated  stories 
with  music  and  other  sound  effects  and  by  those  who 
listened  to  the  stories  without  music  and  other  sound  effects. 

8,  There  will  be  no  significant  dlfferencn  in  the 
retention  of  Information  by  fourth  grade  students  with  high 
reading  abilities  who  listened  to  narrated  stories  with 
music  and  other  sound  effects  and  by  those  who  listened  to 
the  stories  without  music  and  other  sound  effects* 

METHODOLOGY 
Population 

The  subjects  Involved  1n  this  study  were  107  fourth 
grade  students  enrolled  1n  two  elementary  schools  in  a 
public  school  district  in  north-central  Texas.    The  students 
represented  all  fourth  grade  students  enrolled  in  both 
elementary  schools*    The  school  district  encomposses  a 
small  city  and  the  surrounding  rural  area.    The  two  schools 
used  in  the  study  were  selected  because  they  each  had  self- 
contained  classrooms  where  all  activities  could  be  conducted. 

Procedures  for  Conducting  the  Study 
To  eliminate  the  possibnity  that  differences  in  the 
composition  of  treatr^ent  groups  might  bias  the  study,  an 


experimental  design  was  developed  fc^         with  this  study 
which  allowed  the  entire  sample  to  fMnQ^jc^iias   the  exper- 
imental and  the  control  group.     The  fivetf'asses  of  fourth 
grade  students  available  for  the  stud)^  ^^er"^^  a  rb  1  trarlly 
assigned  to  Group  A  or  Group  B  v^itfi  an  attf^mp*  to  have 
approximately  equal  numbers  in  each  gfOup,.  Each  of  the 
five  classes  listened  to  four  taped  stopi^s  ranging  from 
five  to  ten  minutes  In  length.    Each  ^rpy^  recei ved  two 
of  the  stories  presented  in  the  central  f^rifiat  ant  two 
stories  in  the  experimental  format,    thg  niet  r*esult  assured 
the  the  experimental  ar^d  control  groiips  ^^-M^    In  fact^ 
equal*    Group  A  and  Group  B  were  rte^er  cow'P^^^^  against 
one  another;  only  the  treatment  sco^^es  ^er  ^  CQmpared,  The 
two  audio  presentation  formats  utilized  3^  tha  stydy  weie: 

1.  Four  stories  were  narrated  afirt  ye  corded  on  monaral 
cassette  for  playback, 

2.  The  four  stories  were  revised       ^if^cl  uc  music 
and  sound  effects  in  a  radio  drama  f ohnii^ -iliou gh  the  narra- 
tion remained  unchanged* 

The  study  was  conducted  during  the  nior^th  of  May,  1978, 
at  the  same  time  each  day.    The  stqdi^nt^  ^^re  informed  at 
the  beginning  of  the  first  day  that  thfij^  ^rj^^i    be  listening 
to  four  stories^  one  each  day  for  fowr  would  be 

asked  to  answer  a  twenty  Item  mul tf ple^^f^p^ce  test  over 
each  of  the  stories. 


Two  weeks  after  listening  to  the  fourth  storys  all 
students  were  administered  a  recall  test  to  measure 
retention  of  InforFnation  from  the  stories.    The  recall 
test  was  composed  of  ten  inultiple-cholce  questions  selected 
from  each  of  the  fou^  comprehension  tests*    Thus  the  recall 
test  was  a  forty  iteni  test*     No  discussion  of  the  stories 
was  conducted  between  the  time  of  the  first  tests  and  the 
final  recall  test*    Only  the  scores  of  those  students  who 
were  present  on  all  four  days  and  took  the  recall  test 
were  included  in  the  final  analyses. 

The  week  preceding  the  administration  of  the  listening 
comprehension  tests,  all  subjects  in  the  study  had  been 
administered  the  California  Test  of  Basic  Skills  {CTBS) 
along  with  all  students  in  the  school  systern*     The  Reading 
Vocabulary  and  Reading  Comprehension  scores  were  coniblned 
in  the  analyzed  data  to  give  a  total  "Reading"  score 
expressed  as  a  "National  Percentile*'  score.     The  "National 
Percentile"  score  was  utilized  to  divide  the  total  subject 
population  into  three  groups  of  approximately  equal  size 
representing  the  upper^  mfddlei  and  lower  thirds  of  the 
test  population. 

Collection  of  the  Data 
Each  subject  had  test  scores  from  each  of  the  four 
comprehension  tests  and  the  four  part  recall  test.    The  tape 
treatments  that  each  student  received  were  recorded  and 


utilized  to  separate  scores  into  "music  and  sound  effects" 
and  ''narration  only"  columns*    The  net  result  was  a  total 
of  two  scores  for  each  subject  for  each  of  the  two 
treatments.     There  were  thirty-six  subjects  in  the  high 
reading  category^  thirty-seven  subjects  in  the  mediuni 
category^  and  thirty-four  subjects  in  the  low  category* 

Treatment  of  the  Data 

The  data  for  this  study  were  obtained  from  the  four 
twenty  Item  multiple-choice  tests  for  comprehension,  the 
forty  Item  multiple-choice  test  for  recall,  and  the  CTBS 
National  Reading  Percentile,    The  data  from  the  coniprehen- 
slon  tests  and  the  recall  test  ware  analyzed  separatily 
utilizing  the  same  procedures .     The  data  from  the  conipre- 
hension  test  scores  were  analyzed  by  a  two  factor  analysis 
of  variance  with  repeated  measures.    When  an  overall 
significant  difference  was  found  betvieen  groups^  a  multiple 
comparison  test  was  run  to  determine  which  pairs  of  treat- 
ment levels  were  significantly  different.    Within  each 
reading  level  groups  an  analysis  of  va    ance  test  was 
conducted  to  determine  if  a  significant  difference  in 
treatments  was  found*    The  same  procedures  were  used  in  the 
analysis  of  the  data  from  the  recall  test*     The  level  of 
ilgnificance  for  this  study  was  the  ,05  level. 

The  statistical  data  was  analyzed  by  the  Computer 
Center  at  North  Texas  State  University  and  is  presented 


in  table  form.  From  the  findings,  conclusions  were  drawn, 
educational  implications  stated,  and  recommendations  made. 


PRESENTATION  OF  THE  FINDINGS 
The  cell  data  for  the  comprehension  test  scores  are 
recorded  in  Table  I.    The  population  is  divided  into  highs 
medium,  and  low  reading  ability,  and  Includes  scores  for 
the  total  population.     In  addition  to  the  number  of  subjects 
(N)  in  each  cell,  the  mean  (M)  and  standard  deviation  (SD) 
for  each  treatment  combinations  and  the  combined  total 
treatment  scores  are  included,, 


TABLE  I 

NUMBER  OF  SUBJECTS,  MEANS,  AND  STANDARD  DEVIATIONS  FOR 
THE  COMBINED  SCORES  OF  COMPREHENSION  TESTS  AND 
THE  COMBINED  MEANS  OF  THE  MAIN  VARIABLES 


Readi  n  g 
Group 

Music  and 
Sound  Effects 

Narration 
Only 

Group 
Means 

High 

N 
M 

SD 

36 

35. 1389 
2.9096 

36 

34.6667 
3.3722 

34. 9028 

Medl  uni 

N 
M 

SD 

37 

31.6486 
4.4921 

37 

29 .8649 
5 .2766 

30, 7568 

Low 

N 
M 

SD 

34 

24. 1471 
7.6441 

34 

21 .7647 
8.0756 

22.9559 

Total 

N 
M 

SD 

107 

30.4393 
6. 9678 

107 

28.9065 
7.8398 

A  comparison  of  the  mean  scores  for  both  treatments 
within  each  group  and  for  the  total  population  yield  a 
higher  mean  score  for  the  music  and  sound  effects  treatment 
than  the  narration  only  treatment.     It  should  also  be  noted 
that  the  mean  scores  for  both  treatments  wert  ir'gher  for  the 
high  reading  group  and  lowest  for  the  low  v^eiuJing  group. 


TABLE  II 


NUMBER  OF  SUBJECTS.  MEANS,  AND  STANDARD  DEVIATIONS  FOR 
THE  COMBINED  SCORES  OF  THE  RECALL  TEST  AND  THE 
COMBINED  MEANS  OF  THE  MAIN  VARIABLES 


Readi  ng 

Music  and 

Narrati  on 

Group 

Levt  1 

Sound  Effects 

Only 

Means 

N 

36 

36 

High 

M 

18.0278 

17.0556 

17.5417 

SD 

1.6985 

2.8380 

N 

37 

37 

Medi  urn 

M 

15.64Se 

14. 3514 

15. 0000 

SD 

3.0507 

3.2678 

N 

1 

34 

Low 

M 

10.9706 

10.0000 

10.4853 

SD 

4.7064 

4.4107 

N 

107 

107 

Total 

M 

14.9626 

13.8785 

SD 

4.4147 

4.5468 

The  cell  data  for  the  r^ecall  tests  are  given  In  Table 
II.     Againp  the  population  Is  divided  Into  high*  medium, 
and  low  reading  ability  groups^  and  Include  scores  for  the 
total  population.     In  addition  to  the  number  of  subjects  (N) 
in  each  cell^  the  mean  (M)  and  standard  deviation  (SD)  for 
each  treatment  combinations  and  the  combined  total  treat- 
ment  scores  are  included. 

As  with  the  comprehension  test  scores^  a  comparison  of 
the  mean  scores  for  both  treatrnents  within  each  group  and 
for  the  total  population  yield  a  higher  mean  score  for  the 
music  and  sound  effects  treatment  than  the  narration  only 
treatment.    Again j  the  total  means  for  the  high  reading 
group  was  highest  and  the  low  reading  group  lowest. 

In  order  to  deternilne  If  the  observed  differences 
between  the  treatments  and  between  the  groups  were  signifi- 
cantly different^  the  experimental  data  presented  1n  Table  I 
and  Table  II  were  analyzed  by  two  factor  analysis  of  variance 
with  repeated  measures.     The  results  are  shown  in  Table  III 
and  Table  IV. 

The  results  of  the  two  factor  analysis  of  variance  with 
repeated  measures  for  listening  comprehension  is  presented 
in  Table  III,    The  analysis  of  effects  between  the  reading 
groups  with  two  degrees  of  freedom  yielded  a  sum  square  of 
5242.9455,  a  mean  square  of  2621.47275,  and  F  Ratio  of 
49.35601,  and  a  P  value  of  0*0000  which  means  the  difference 


is  significant  at  the  *01  level.    The  analysis  of  the  treat- 
ments with  1  degree  of  freedom  yielded  a  sum  of  squares  and 
mean  squares  total  of  127,73243,  an  F  Ratio  of  13,56032, 
and  a  P  value  of  0.00037  which  is  significant  at  the  .01 
level  also.    The  analysis  for  treatment  interaction  with  2 
degrees  of  freedom  yielded  a  sum  of  squares  scort'  of  34,  00209  , 
a  mean  square  of  17.00104,  and  an  £  Ratio  of  1.80486,  and 
a  P  value  of  0*  16950  which  Is  not  significant  at  the  .05 
level . 


TABLE  III 

TWO  FACTOR  ANALYSIS  OF  VARIANCE  WITH  REPEATED  MEASURES  OF 
THE  EFFECTS  THAT  MUSIC  AND  SOUND  EFFECTS  AND  NO  MUSIC 
AND  SOUND  EFFECTS  HAVE  ON  LISTENING  COMPREHENSION 


Source 

DF 

Sum  of  Squares 

Mean  Squares 

£  Ratio 

P 

Between 
Readi  ng 
Groups 

2 

5242  94550 

2621  47£75 

49.35601* 

0.00000 

Treatment 
Effects 

1 

127. 73243 

127. 73243 

13.56032* 

0.00037 

T  reatment 
In  teract 1  on 

2 

34.00209 

17.00104 

1.80486 

0.16950 

Within  Cell 
trror 

104 

979.63595 

9.41958 

Total  109  6384.31597 


*    Stati  stiCfil  ly  significant  at  P  0.05 
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TABLE  IV 


TWO  FACTOR  ANALYSIS  OF  VARIANCE  WITH  REPEATED  MEASURES  OF 
THE  EFFECTS  THAT  MUSIC  AND  SOUND  EFFECTS  AND  NO  MUSIC 
AND  SOUND  EFFECTS  HAVE  ON  RECALL 


S  n  urrp 

W  M  1   W  w 

u  r 

nean   Dqud res 

r    KQ  t 1 0 

Between 
Readi ng 
Groups 

2 

1819.95141 

909.97570 

46.80713* 

O.OOOOO 

Treatnient 
Effects 

1 

62.32917 

62.32917 

14.25180* 

O.O0027 

Treatnient 
Interaction 

2 

1.26113 

0.63056 

0.14418 

0.86590 

Within  Cell 
Error 

104 

454.83627 

4.37343 

Total  109  2338.37798 

^    Statistically  significant  at  P  0,01 


The  results  of  the  two  factor  analysis  of  variance  with 
repeated  measures  test  for  recall  is  given  in  Table  IV.  The 
analysis  of  effects  between  the  reading  groups  with  two 
degrees  of  freedom  yielded  a  sum  of  squares  of  1819.95141,  a 
mean  squares  of  909  *97570,  an  £  Ratio  of  46.80713  anc:  a  P 
value  of  0.0000  which  is  significant  at  the  ,01  level.  The 
analysis  of  the  treatments  with  one  degree  of  freedom  yielded 
a  sum  of  squares  score  of  62,32917,  a  mean  square  score  of 
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62.32917,  an  F  Ratio  of  14.25180,  and  a  P  value  of  0.00027 
which  is  significant  at  the  .01  level.     The  test  for  inter- 
action with  two  degrees  of  freedom  yielded  a  sum  of  squares 
score  of  1,26113,  s  mean  squares  score  of  0.63056,  and  £  Ratio 
of  0.14418,  and  a  P  value  of  0.86590  which  means  interaction 
is  not  significant  at  the  .05  level. 

To  deterniine  how  each  reading  level  reacted  to  the 
separate  treatments  a  single  factor  anaiyais  of  variance  with 

TABLE  V 

ANALYSIS  OF  VARIANCE  OF  THE  EFFECTS  OF  MUSIC  AND  SOUND  EFFECTS 
AND  NO  MUSIC  AND  SOUND  EFFECTS  HAVE  ON  THE  LISTENING 
COMPREHENSION  OF  HIGH,  MEDIUM,  AND  LOW 
READING  GROUPS 


1  Source 

DF 

Sum  of  Squares 

Mean  Squares 

F  Ratio 

P 

High 
Readi  ng 
Group 

1 

4.01389 

4.01389 

1.00495 

0.30470 

Medi  um 
Readi  ng 
Group 

1 

58.86486 

58.86486 

5.11701* 

0.02983 

Low 

Reading 
Group 

1 

96.48529 

96,48529 

7.30254* 

0.01079 

*    Statistically  significant  at  P  0,05 


repeated  measures  was  carried  out  for  each  reading  level 
for  the  listening  comprehension  tests  and  the  recall  test* 
The  results  are  given  in  Table  V  and  Table  VI. 

The  data  presented  in  Table  V  is  for  the  Independent 
analysis  for  the  three  reading  levels  of  listening  compre- 
hension*   The  high  reading  group  sum  of  squares  and  mean 
square  is  4.01389,  the  £  Ratio  is  1,08495,  and  the  P  value 
is  0.30470  which  is  not  significant  at  the  ,05  level.  The 
medium  reading  group  has  a  sum  of  squares  and  mean  square 
score  of  58.86486^  an  £  Ratio  of  5.11701^  and  a  P  vahie  of 
0,02938  which  is  significant  at  the  .05  level.    The  low 
reading  group  has  a  sum  of  squares  and  mean  square  score 
of  96.48529,  an  F  Ratio  of  7,30254,  and  a  P  value  of  0,01079 
which  is  significant  at  the  .05  leveK 

The  data  presented  in  Table  VI  is  for  the  independent 
analysis  for  the  three  reading  levels  on  the  recall  test. 
The  high  reading  group  sum  of  squares  and  mean  square  total 
is  17,01389,  an  F  Ratio  of  5.54012,  and  a  P  value  of  0.02433 
which  is  significant  at  the  ,05  level.    The  medium  level 
received  a  sum  of  squares  and  mean  square  score  of  31.13514, 
an  £  Ratio  of  3.23262,  and  a  P  value  of  0.08134  which  is 
not  significant  at  the  ,05  level. 

The  Scheffe  F  values  for  comparisons  among  reading 
groups  for  listening  comprehension  and  recall  tests  are 
given  In  Table  VIl.    The  comparison  of  group  means  for  the 


TADLf  VI 


ANALYSIS  OF  VArUAMCE  OF  THE  EFFECTS  OF  MUSIC  AND  SOUN 
EFFECTS  AND  NO  MUSIC  AMD  SOUND  EFFECTS  HAVE  ON  THE 
RECALL  OF  HIGH,  MEDIUM.   AND  LOW  READING  GROUPS 


Source 

OF 

S u r ii  of  Squares 

Mean  Squares 

F  Ratio 

P 

High 
Readi  ng 
Group 

1 

17. 01389 

— — — ■'-  — ^ 

17.013S9 

5,54012^ 

0.02433 

Mg  di  uni 

Reading 

Group 

1 

31. 135  14 

31.  13514 

6.09C13* 

0.01843 

Low 

Reading 
Group 

1 

16,01471 

16.01471 

3.23262 

0.08134 

*    Statistically  significant 

at  P    0.0  5 

high  and  nie 

di urn  reading  groups   gives  a  Scheffe's  F 

test 

value  of  5,90  53  for  the  lis 

t  e  n  1  n  g  c  o  ni  p  r  e  h  e  n  s  i  o  n  t  e  s 

t  and  a 

6. 0632 

V  a  1  u  ( 

3  for  the  recal 1 

test;  both  were  significant  at 

the   .05   1  e vt 

:f  1  .     Fhe  compdr i 

son  0 f  liiG d i  uin 

and  1 0  \'/  reading 

g  0  u  p  $ 

gi  ve': 

\  fin  F  value  of 

20.3004   For  11 

s  t  e  n  i  n  g  comprehension 

and  18. 

5  755 

for  re ca 1 1 ;  hot 

1  0  f  \'/  Ii  i  c  h  a  e 

significant  at   t fi c 

.05  level , 

The  comparison  of  the  high  an 

1   1 ow  read  1 n g 

groups 

g  i  V  G  s 

an  F  value  of  ' 

16,9  8  81   for  listening  c  o 

np  re  [ions  1  on 

and  44.7B46  for  recall;  both  again  are  significant  at  the 
,05  level. 
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TABLE  VIl 

SCHEPFE'S  F  VALUES  FOR  COMPARISONS  AMONG  READING  GROUPS 
FOR  LISTENING  COMPREHENSION  AND  RECALL  TESTS 


Group  Comparison 

F 

Li  s teni  ng 
Comprehens  i  on 

Value 

Recal 1 

HI gh-Medi  um 

5.3053  * 

6.0632  * 

Medl um-Low 

20.3004  * 

18.5765  * 

H 1  g  h - L  ow 

46.9881  * 

44.7846  * 

*    statistically  significant  at  P  0,05 


SUMMARY  OF  FINDINGS,  CONCLUSIONS,  IMPLICATIONS 
AND  RECOMMENDATIONS 

Findings 

1.     It  was  hypotheslEad  that  there  would  be  no 
significant  difference  in  the  listening  comprehension 
scores  of  fourth  grade  students  who  listened  to  narrated 
stories  with  music  and  sound  effects  and  those  who 
listened  to  the  same  stories  without  music  and  sound 
effects.     The  analysis  of  the  data  reveals  that  the  scores 
on  the  listening  comprehension  tests  were  significantly 
higher  for  those  students  who  listened  to  the  stories  with 
music  and  sound  effects. 


2 (t/avas  hypothesized  that  there  would  bej  no 
:i  i  gn  1  f  i  can  t '  d  i  f  f  eren  ce  in  the  listening  coinp  roll  ens  i  on 
scores  of  Fourth  grade  students  with  low  reading  abilities 
who  listened  to  stories   recorded  with  music  and  sound 
effects  and  stories  without  music  and  sound  effects.  The 
analysis  of  the  data  revealsthat  the  scores  on  the  listening 
coinprehens  1  on  tests  were  significantly  higher  for  those 
students  who  listened  to  the  stories  v/1  th  music  and  sound 
ef  fee  ts  . 

3  .     I  t  was  )i y p 0 1 h e s i  z e d  that  t ti e r e  w o u  1  d  be  no 
signified nt  difference   in  the  listening  comprehension 
scores  of  fourth  grade  students  with  niediuni  reading 
abilities  who  listened  to  stgries   rec'^     -      nth  music  and 
sound  effects  and  stories  without  mu  n  ^:  r.ic  sound  effects. 
The  analysis  of  the  data  reveal   thot  the  scores  on  the 
listenirKj  coiri{.  rehen  s  i  on  tests  were  significantly  highur 
for  those  studt'nts  who  listDnod  to  the  stories  with  music 
and  sound  eifects. 

4,     It  was  hypothesized  that  there  would  be  nu 
significant  difference  in  the  listening  comprehension 
scores  of  fourth  grade  students  with  high  reading  abilities 
wlio   listened  to  stories  recorded  wi  tli  fiiusic  a  fid  sound 
effects  and  stories  without  music, and  sound  effects.  The 
analysis  of  the  data  reveals  that  tha  scores  on  the  listefiing 
comprehension  tests  were  higher  for  those  studcjnts  who 
listened  to  the  stories  with  music  and  sound  effects,  but 
the  differ*'-'^  was  not  significant  at  the   ,05  level. 


5.     It  was  hypothesized  that  there  would  be  no 
significant  difference  In  the  retention  scores  of  fourth 
grade  students  who  listened  to  narrated  stories  with  music 
and  sound  effects  and  those  who  listened  to  the  stories 
without  music  and  sound  effects.     The  analysis  of  the  data 
reveals  that  the  scores  on  the  recall  test  were  significantly 
higher  for  those  students  who  listened  to  the  stories 
with  music  and  sound  effects. 

6.  It  was  hypothesized  that  there  would  be  no 
significant  difference  in  the  retention  scores  of  fourth 
grade  students  with  low  reading  abilities  who  listened 

to  narrated  stories  with  music  and  sound  effects  and  those 
who  listened  to  the  stories  without  music  and  sound  effects. 
The  analysis  of  the  data  reveal  that  the  scores  on  the 
recall  test  were  higher  for  those  students  who  listened  to 
the  stories  with  music  and  sound  effects,  but  the  difference 
was  not  significant  at  the  .05  level. 

7.  It  was  hypothesized  that  there  would  be  no 
significant  difference  in  the  retention  scores  of  fourth 
grade  students  with  medluni  reading  abilities  who  listened 
to  narrated  stories  with  music  and  sound  effects  and  those 
who  listened  to  the  stories  without  music  and  sound  effects. 
The  analysis  of  the  data  reveals  that  the  scores  on  the 
recall  test  were  significantly  higher  for  those  students 
who  listened  to  the  stories  with  music  and  sound  effects. 


8.     It  was  hypothesized  that  there  Wpgi  ^  be  no 
significant  difference  in  the  retention  Scor^^^  of  fourth 
grade  students  with  high  reading  le\rel  $b]  ]\  ties  who 
listened  to  narrated  stories  with  musTc  and  sound  effects 
and  those  who  listened  to  the  stories  i^i  tf|     t  mus  i  c  and 
sound  effects.     The  analysis  of  the  data  rev        that  the 
scores  on  the  recall  test  were  si  gnl  ff  cantly  higher  for 
those  students  who  listened  to  the  st&rt&s  ^  music 
and  sound  effects, 

Concl ys 1 ons 

Based  on  the  data  presented  in  this  stuj^  ^ind   the  limit- 
ations stated,  the  following  conclusions  hQV#*been  drawni 

1-    The  addition  of  music  and  sound  ^ffpccts  to 
recorded  presentations  increases  the  Us t^^^i^g  comprehension 
of  fourth  grade  students, 

2*     Information  from  a  recorded  pres^^t^tion  with 
music  and  sound  effects  can  be  remenibeTed  lorfiper   by  fourth 
grade  students  than  the  same  inforination  pre^^ntad  without 
music  and  sound  effects, 

3.     Fourth  grade  students  with  tilgti^  ffie^ium,   or  low 
reading  abilities  comprehend  and  recall  recor^^^d  Information 
presented  with  music  and  sound  effects  to  a  g  reater  degree 
than  when  the  Information  is  presented  wtttioy  t  music  and 
sound  effects. 


4.  Although  the  scores  for  the  high  reading  group 
were  not  significantly  higher  for  listening  comprehension, 
for  those  listening  to  the  music  and  sound  effects  stories^ 
their  mean  scores  were  higher*     After  two  weeksj  those  who 
listened  to  the  music  and  sound  effects  did  score  signifi- 
cantly higher.     The  research  indicates  students  who  are 
better  readers  are  also  better  listeners     so  a  significant 
difference  might  not  appear  immediately  after  a  listening 
exercise  but  s .merge  after  a  period  of  time,  thus  explaining 
the  significantly  higher  recall  scores  for  the  music  and 
sound  effects  group. 

5,  The  low  reading  group  reacted  completely  opposite 
the  high  reading  group  by  showing  a  significant  difference 
in  favor  of  music  and  sound  effects  immediately  after 
listening  to  the  stories  but  failed  to  score  significantly 
higher  on  the  recall  test.     The   low  readers,  as  the  research 
also  indicates,  are  poorer  listeners  and  even  though  the 
music  and  sound  effects  may  have  made  an  immediate  dlfferenc 
in  comprehension,  this  difference  was  not  great  enough  to 
make  a  significant  difference  after  a  two  week  delay  in 
recall  of  information, 

Impl i  cati  ons 

The  findings  of  this  study  indicate  the  fnclusion 
of  music  and  sound  effects  in  taped  programs  for  fourth 
grade  students  increases  their  ability  to  recall  information 


presented.     Both  long  and  short  term  memory  of  the  rnaterial 
is  increased.     Although  the  data  is  not  conclusive  for  all 
leve'is,  there  is  a  strong  indication  that  the  addition  of 
music  and  sound  effects  to  recorded  material  helps  students 
at  all   levels  of  their  language  arts  development* 

Why  the  ad d it i on  of  music  and  sound  effects  should 
cause  such  an  increase  1n  comprehension  and  retention  is 
not  totally  understood,  but  this  and  other  research  indicate 
several  possibilities.     The  music  and  sound  effects  could 
aid  in  drawing  attention  to  specific  informatinn  being 
presented.     The  music  and  sound  effects  may  mask  out  other 
possible  distractions  that  nay  be  occurlng  in  the  listening 
environment  and  allow  the  :  cuc.it  to  pay  closer  a^  tion. 
The  music  and  sound  effects     :y  help  creat  a  ment   i   ih  ge 
of  what  is  being  presented.     It  may  be  that  music  and  soun^- 
effects  play  an  Important  part  in  helping  to  visualize  the 
message  by  adding  ''color'*  to  the  recorded  message. 

When  students  1n  this  study  were  asked  which  programs 
they  vould  like  to  listen  to  a  second  times  they  over- 
whelmingly requested  the  stcries  they  had  heard  with  music 
and  sound  effects.     What  some  educators  and  radio  specialists 
have  been  saying  for  decades  seems  to  be  supported  by 
research:     music  and  sound  effects  added  to  audio  progran: 
do  make  a  difference* 

It  seems  fair  to  assume  that  today's  children  have 
grown  accustomed  to  audio  material  being  presented  in  a 

er|c  4u..  '^^^ 


professional,  well  produced  format  and  tend  to  listen  more 
effectively  to  programs  produced  1n  a  more  sophisticated 
manner.     Because  children  are  bombarded  by  sights  end 
sounds  from  numerous  sources,  they  may  have  become  very 
selective  in  directing  their  attention.     Sound  sources 
that  do  not  challenge  or  eKcUe  thern  tend  to  be  disregarded, 
while  sound  sources  that  are  exciting  to  their  ears  are 
listened  to  more  concertedly. 

The   iinpl  icatlons   of  this   stuny   should  .erv« 
what  of  a  warning  for  teachers  and  other  educators-^that 
care  should  be  taken  In  selecting  and  preparing  audio 
materials  for  use  with  children.     Whan  children  have  tape 
recorders  at  home  with  which  to  play,  there  Is  little 
interest  in  the  technology  itself  to  maintain  interest  In 
listening  to  a  boring  tape,  espscially  when  It  is  competing 
with  eight  tracks  and  stereo  cassettei   with  all   types  of 
stimulating  sound:;.     Besides  being  interested  In  the  content 
of  a  recorded  p-cgram,  the  quality  of  the  production  itself 
must  be  Investigated  by  those  selecting  materials  for 
children.     If  the  teacher  Is  not  selective,  the  students 
will  be. 

The  educator  does  not  havi^   t-o  facp  thp  full   l)runt  uf 
the  selection  task  dlone.     ProduCfMs  uf  -mIu,  „  t.  i  ona  ! 
materials  must  share  responsibility  for  the  materials  theiy 
produce.     As  funds  for  educational  materials  become  more 
difficult  to  secure,  the  quality  of  those  materials  pur- 
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chased  must  provide  the  best  quality  for  the  money.  Ac. 
accountabnity  for  what  educators  teach  and  children  learn 
continues  to  be  stressed,  the  producers  of  educntional 
m^iterials  must  be  held  more  accountable  also. 

Recommenda  ti  ons 
While  the  findings  of  this  study  support  the  claim 
that  music  and  sound  effects  incritase  the  ability  of 
fourth  grade  ^t^^dents  to  listen  and  comprehend  information 
i^resented  on  recnrdings  more  effectively,  several  questions 
remain  to  be     nsweved      This  study  dealt  mainly  with  the 
recall   of  sp:^cific  f c  :ts  from  the  stories;  higher  level 
mental  skills  such  as   Inference  and  critical  thinking  were 
not  tested.     F.iture  research  should  include; 

1,  ^.tudies  to  Investigate  the  effect  of  music  and 
sound  effects  on  the  inference  and  critical  thinking 
abilities  of  students; 

2 ,  studies  to  1 n ves  ti gate  the  pos  s  i  bi 1 i  ty  th  at 
changes  in  the  +ype  of  music  and  sound  effects  can  change 
perception  of  what  is  happening  in  the  story  or  recorded 
message ; 

3,  studies  to  investigate  the  effect  on  attitude 

and  possible  attitudinal  changes  resulting  in  the  selection 
of  the  music  and  sound  effects; 

4,  studies  to  investigate  the  possiule  masking  effect 
of  over-'use  of  music  and  sound  effects. 
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Children's  UndersCancling  of  Implied      tion  Cues 
Ronald  A,  SaieC 

Extrapolating  from    'lagat's  cheory  of  cognitive  development  it  can  be  argued 
that  graphic  convent:..  .1,;  sue::  as  flow  lines  and  vibration  marks  used  by  artists 
to  depict  motion  woui'ii  ii/r      little  or  no  meaning  to  pre--school  children,  but  that 
an  action  posss  being  niart^  like  real  human  posture,  would  be  a  syubol  within  the 
grasp  of  understanding  of  children  in  this  group.     By  age  seven,  most  children 
should  be  capable  of  understanding  arbitrary  graphic  symbols,  and  by  age  eleven 
virtually  all  should  be  capable. 

The  kind  of  object  upon  which  a  motion  cue  is  said  to  act  might  also  affect 
perception  of  movcirent*     Human  figures,  for  example,  might  appear  to  be  more  likely 
to  be  in  motion  than  abstracted  geomecric  shapes.    An  inherent  motiori  factor  may 
also  be  said  to  reside  wlchin  an  object,  such  that  it  psychologically  appears  to 
be  more  active  or  more  static-^^running  figures,  ellipses,  and  rectangles  may  be 
seen  as  more  active  than  standing  figures j  circles  or  squares • 

An  experimental  study  was  conducted  in  an  attempt  to  determi.'    how  children 
--^fourj  seven,  and  eleven  years  old^-perceive  implied  motion  cues  in  combination 
vlth  a  variety  of  objects,     Ninecy  children ^  30  in  each  age  groups  were  shown 
24  pictures,     Tnere  were  three  kinds  of  objects,  each  with  an  inherent  active  and 
static  d:^.menslon*     E  .c.i  of  thesa  six  pictures  appeared  with  one  of  three  motion 
cues,  and  alone,    Tlie  number  of  each  of  the  picture  types  identified  as  moving 
by  each  subject  was  determined.    An  age  x  object  x  activity  of  object  k  motion 
cues  repeated  measures  A^^OVA  was  used  to  evaluate  the  data. 

^U.l  miin  ef feces  and  all  interactions  wer^i  significant.     It  was  concluded 
that  children  at  different  stages  of  development  differ  in  their  interpretation 
of  certain  graphic  conventions.     Illustrators,  editors^  and  publishers  should 
take  into  account  the  pictorial  literacy  of  an  intended  audience  when  they  design 
materials  to  convey  motion  information. 
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*'0f  all   Lhe  sensuB,    Lrui4L  only  tha  sensL:  of  ^ight." 

Ar  iatot  Lo  ,   ^'  -  w)hysics 

Li.  ttlu  niore   than  a  duca'^hj  ago,   Plnrninq    ^''iG6)  observed 
that  in   the  rnidst  of  new  trGnds  emuroinn   iri  aducational  trach-^ 
nology^   the  coninionplacQ  print  maclia  GOntinucd  largGly  un-- 
challanged  as   the  most  widely  used  Gducational  resource   (p.  299) 
He  also  found  a  pro!  i  f  era  t .; -.      of  i  llus  tra  t  inns  in  textbooks, 
one  illustration  appearinu  avury  1,58  pages  of  40  oightb  crrade 
tOKts  sampled   (p.    317) .     Caaual  obsoryation  of  textbook  offering 
in  children' n  jiterdture   today  reveals   that  the  situation  now 
appears   to  b-  niuch  as  it  was   then^   and  ono  CQntinues  to  wonder 
along  with  Fleming  what  osaential  role  illustrations  serve; 
they  add   to  the  size  and  costy  of  printed  materials,   but  is 
there  any  evldcncQ   that  pictures  add   to  a  child's  understanding 
of  the  mavt^^;    before  him? 

GroppcL    ■19G2)    obsorvod  that  much  loss  is  known  about  how 
people  respond  to  visuals   than  about  how  peoplQ  respond  to 
words.     Art  students  are  taught  what  Uondis    (1973)   called  tha 
basic  elements  of  visual  commun  ica  tion^-=do  t ,    line,  shape, 
direction,    tune,   CDlor,    toxturu,   dimensioni-   scale,   and  move- 
ment  (pp.    39-66) --^but  ^unyun    (1970)    asked  whether  cnlldren 
have  acquired  an  under n tand inn  of  these  eloments,   and  if  sO/ 
whether   they  are  acquired  thrt.^ugh   f:alent  or  thirough  training 
(p.    3).     The  prosent  wrj.ter*y  survey  u"   ^nci  .Literatuie  has 
found  it  wanting  i:i  qupsticMKi  and  a::sweiB  nbtjut  h^^*'  people-- 
cirildren  and  aciul  ty--pQrct!i  vo  cii^  J   le;'  /n   !:rom  noir-vcrha  1 , 


non-moving  viaual  atimuli. 

The  problem  Cor  mcaaayij  dosigiicrs  was  statsad  by  Fleming 
as   foliowH:      "IL  is  improbable   that  an  instructional  system 
can  apLJroK  jma  tu  optimum  pof  f  otrinnnccj  unlctis   tliG  designer  of 
that  syatom  ia   fully    Lnforinnd  on   the  relovant  characteristics 
of   the  syy  tofii  cDiiiponont  options.     Pictorial  stimuli  will 
be  one  of   tho  system  componant  options  the  designer  will 
select   to  nifjQt  certain   systom  specifications"    (1966,   pp  -318), 

It   IS  the  aim  of   this  iiapor  to  examinG  a  small  pai  ■  - 
this  problem  within   thcj  aroa  of  pictorial  communication. 
The  investicjd  ti'jn  carried  out  here  was  sug.^^isted  by  a  doctoral 
dissertation  by  Sa t tor thwai to   (1965),    "An  investigation  of 
sclcctod  cues  for  implying  motion  in  non-motion  media." 
It  was  Sattcrthwai  to '  H  assiunption  that  graphic  devices 
(called  "impliod  motion  cuus")   conumjnly  used   in  two- 
dimQnsional  print  media  by  artists  and   i  .1  Iuh  tra  tor  s 
sucGssfully  tranidinit  isome  motion  information   to  the  perceivers 
of   those  graphic  duvicos.     Ho  then  oGlcctod  and  manipulated 
three  such    iiiipliod  motion  cuusS    and   asked   "Do  all  of  the 
iJossibie  variations  of   tht.iie    iniplied  motion  cues  transmit 
the   same  mot  inn    l  n  to  tina  t  io  n  oi'  ,irn   hjor-iD  coinhinations  of  cues 
and  objects  ijoist   .,ai.  Led    f  e  t    the   tr^umai  i  la-Jion  of  certain 
types  of  motion   i nf orma tiun? "    (pp.    20-.  Using  40  subjects, 

all  sLudents  enrollt.d  in  a  qraduato  courua   in  Audio-Visual 
Communica'  ions  at  Indiana  University,,   Satterthwaite '  s  basic 


— "-^^^  2.  Action  line  (((^^3.  Action  P"S'2^^ 
r  definitlanB. )  C  r 
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assumption  thai;  Uhc  cues  suucobb  i:uXly  impliGd  motion  v/as 
upheld  and  Uhoj  £inKWQr  to  his  bnsic  cjuostioji  was  that  cortain 
varjationy  and  combina t ions  of;  cues  atid  objects  did  afeect 
the  ^.'omrr^wi  iuati  i  on  d  i  f:f  ur  :  ^:  i  ally    (jjp.  123-I2fi), 

It    LK  not  surpirisj!         ii  laa-iL  to   this  investigator,  that 


intontiun  of:   tha   imp  .  i.ad  .  cues.     Thoy  uspecially  would 

SGoni  likely   to  have  ..-icDun  to :        such  cues  in  their  lifetimes 
and   to  havo  aL^Hiunod  accurate  noaninqs  to  them   (accuratG  because 
thoir  intorprctat ion  was  congruont  with  tho  dGSignGr's  intended 
meaning).   It   is  not  Burprising,   either,    that  even  t'  group 
of  subjects  did  not  all   interpret   the  vririations  of  tii^  cues 
in  the  samo  way,   because  nioi;iL  of  Sa  ttorthwai  to  ^  s  variations 
(i.e.;    lonu/Hhort,    tiiick/thin,   and  otiiorH*)    do  not  have 
universal  muaninq,   were  never  systematically  tauyhl  or  learned 
by  the  F^ubjoc  s,   and  arr   arbitrary  in  nature.     As  Knowlton  (1966) 


upon  the  concept '.^  having  already  beo!i  attained  by  its  interpreter 
(p.    165).      Fat  and   thin  lines   intended   to  Imply  more  or  less 


an^  lave  to  be  learned  if  otlu-:r  than  idiosyncratic  inierpre  tat  ions 
are   to  be  given . 


StuderP:;^    in  a  graduaf- 


ui  j  ,  -    i  3ual   course  unders  tood  the 


observed,  a 


abili   y   to  signify  a  concept  is  dopendent 


speed,    for  exami-d.e,    are  arbitrary   SLgns  within   this  conte.<t, 


1. 
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Pooplo  are  no,  born  litorato.     Tl.rough  a  oomblnatxon 
of  factors^^develop..ent,  .Kporience,   formal  .ducat ion^^paopl. 
eventually  loarn  to  decoae  v.aual  messages.     The  b.sic 
alternate,  quusaon   to  S  ^  t  tor  thwax  te  ^  e  which  is  to  be 
Posod  then,  do  g,,,,,,,  cff.cLlvely  transmit 

motion  xnfarmat.on7     Put  another  way,   at  what  .tage  of 
devolopn^ont  do  children  acourat.ly  percerve  visual  cues 
designod   to  imply  n^otion   fron.  non^mov.ng  prctorial  stimuli? 

According   to  Piaget^s   theory  of  counitive  dovelopmont 
□  no  eithor  assimilates  now  .information  in  La  an  alroady 
existinq  niuntal   citDuory    ( [  ^Hyniio  log  j  ca  I   Htruc^urG)    as   somG  = 
thing   like  or  uni  i  ku  tKJinollrjng  alraady  unuors^tood,   or  ono 
accommodates  and  changoB  hii^  muntal   s  true  turn-^crea  tes  a 
now  catogory"^^  to  dual  with   iMformation  which  cannot  be  so 
assimilated.      Fur  ihormoro ,   -u.^n  chiJd  jjrc^cood^   through  a 
sorioH  of  sLlUK'S,   -ach  Htau^   rof^rcnon  tud  by   the  presonce 
of  particular  psycho  log  ica  I   sr^.r.    ore^u      .  ■  '  nco  experience 
is  always  changing ,    new  structur^.     :iru  always  ovoiving; 
with   the  roBul  t   tiiat  ddult  structuros  arc  more  sophis ticatod 
than   thoso  avallahli'  to  chi  Uiroiu      giahly  ovolved  adult 
i  n  tul  1  igencu    in    rojn.  oneritcc]   hy  inDVc   n  ^^fu.fn  i  la  t  ioii  into 
^^'KlsMng,   Wei.  J    devolupc^d   rUru.fures  uhercas   learners  at 
earlier  st  '  ;o5i  of  rl .       Lo^^men  t ,   riot  having  yet  ovolvod  the 
nQcosHary  :;tructurus  fuc^saal  at  later  nfag-n  of  cognitive 
dovolapniunt ,   are  prc>no  to  Jin  s  m  tcugire  t    (or  'gnoro) 
i  n  f  orn\a  t  ion  , 

If  a  child  liorcoivaB  a  pioLoriLil  oloment  as  familiar, 
he  will  bo  able   tt    MrcjcoHS   it,    to  mako  sonwo  of  it.     If  on 
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the  oUher  hfind,   ha  j  hi  un^bj.fj   to  r*.  .'ugni^ic  a  p!  -:torial 
eleiTienb^-pQrson,   plricQ,    thinq,   or  ac t ion-- thi  ^  ..i^Lght  bn 
construGd  as  an  inaicntion  of  hii:  picture  i  11  i tfsracy ,  An 
iruplicd  mouioii  cu--,    if  not  within  the  rcinge  oJ:  a  child's 
eKperiencu/    would  ba   incunu:a'<  ?honH  iblc   to  him.     A  vicwar  at 
a  higher  luvul  of   picture   li.  toracy,   I'o:^^  SGSBing  only  partial 
literacy,   would  rnako  nensa  oi;   yuine  i^arts  of  picturos  but 
would  rni  H 1  nterprct    tho  rnciaiiiny  ol:   the  c^ntirG  pictorial 
mcHBayo;    if   a   sinulu  cuo         in  i  ^^i  n  ti;' ri  ^rc  ted  ,  mfjanina 
incomp  La  t^i .      Attained   riicture    Ij.Lc^racy  v/cnild  bo  Qvidon 

by  one's  ab.]  I  i  ty   t^^  corructJ.y   jaiturprot   complcax  visual 
symbols,   ab*;  t  laic  t  i  u  nu  ,    and  uraphic  conventions. 

Considering   the   ev.jonye  [Hiblishurfi   incur  in  creating 
illustrative  niaterialH,    it  v/r^ulri  neuiu   to  be  usnfui   to  th^:im 
to  kncnv  at  whut   ntM'ji.-;  of  ch  ^  ve  J  opnwj  n  t   t!iei.t-  audiencGia  aro-- 
their   level   uf   pie^eir^^    M.  t^e-aey-^-£jo   thnt   tlie  f^ictnrial 
inCorniation  can  be  niadL*   to  \)c  appro^jrintc   to  each  audiencro. 
One  might   dSLurne   that  an  Am-jrican  child,   even  at  age  four, 
will   recognL^e  pjetorial   renieHen  ta  tionr;  ul  a  familiar 
object  becauJK^  ijf  his  r^xf  n  ^  v  i -iico  wjfdi   I  e  U;^  v.i  selon  ,    filn,  and 
booku.      I3ut    Lt   i  ^;  anotlter  nuLLtMi;   to  pi^ediet  v/ith  any 
accuracy  at  wiiat  ago   luj  wild    ! ab'J.e   to   jjif^a;  iiiotion  fhcm 
two-dinmn^3  ional  nic^  t  i  on  con  v?  -  n  t  i  ons  .      T  t   F^eeins   pl^niH.i  bl  a  , 
howovor,    tiiat  as   chilfJren  gc^t  nldor   tliey  recognize  Lao 
intended  meaning  fjf  certnin  cuey;    tJiat  certain  motion  cues 
will  be  perceived  more  often  ars  containing  motion  information 
by  children  as   thuy  mature};   and   that  inherent  charac  tor  i:?i  t  ics 
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(such  as  snape  ,^i.d  lif  e-l  LkenesB)   of  nbj(5ats  purported  to 
be  in  motion  ^iffect  the  corrmiuiiicativa  power  of  motion  cues 
as  perceivGd  by  children  at  different  stages  of  development- 
The  study  undertaken  by   the  current  writer  investigates 
the  problem  of  v/nen  and  how  children  at  different  stages  of 
development  understand  implied  motion  cues.     The  current 
vehicle  of  presentation  precludes  a  more  thcrough  review  of 
the  relevant^   albeit  limited,   literature.     For  additional 
explication  of  Piaget's  theory  relevant  to  the  current  exlora- 
tion,   see  Ginsberg  &  Opper    (1969),   Phillips    (1975),  Rohwer, 
Ammon  &  Cramer   il974),   and  Wadsworth    (1971).     Research  con- 
ducted by  the  Hign/Scope  Educational  Research  Foundation 
(Banet,   ut.   al,,   and  riederveld  &  Thomson,   1972)    into  children's 
undertanding  of  representative  material  and  the  implications 
for  learning  is  usefulr   and  Friedman  &  Stavenson's  study  of 
developmental  clianges  and  thoir  affect  on  understanding 
of  implied  motions  cues   (  1975)   provides  an  alternative  ex-- 
periraental  approach  utilizing  an  interesting  array  of  stimulus 
niaterials,   and  v;as  most  useful  in  developing  the  current  study. 
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This  ptud/  wm   clGBign^a  %)   fi  ncl  out  how  chi.lclren  - -4 , 
7/   ancl   11  /ac^rs  o Ja^-yo^ceiv^Q   y^luul^ecl  iciiplied  rrtotion  cms. 
The  inipliQdl  tnotioii  cues   seloc^e^l  £  or  thi^  studi^    were  tlie 
flow  1  i  na  ,    th  e      bi; at:  io n  nncii^k  ^  ^nd     he  b  Jur  -    ria e  di:  a v  ing  s 
usod  ^uVi^  ^G].  G^tied  by   thr^^.   jkid^ss    (  cir^apSiic    arti.::':^)  accord- 
incj  to   tvro  Cirltciria;  th^y  hscl   t*^  b  q  slniiJar    to  ^hc  postural 
GhcirnctarH  and  cartaon  dc\^iCG^  ^s^^cl   irt  F^iecliimn    rind  S  tevenson 
(  1  9  7  5),  aii^  tha^  b^d  to   cohv^e^  no  t  Jo  n   In  formation  to  tbem 
(I.e.,    to  th^  j  - 

lmplic2d  3^icrti Oh   cue^*  .  ,thQs^  g  rapKic  de\/ici3^  comraonly 


UR^cl   to  repre^Gnti  motion  in  non- 


inoti^n  jiicdia    (i.s-/  ^liildr  ^r's 


Flow  line, 


.       Qjr^iPh  ia  aovJce    used^  to  isnply 

Bmedi    it  consists  off  even  ,  clearly 


sp^c^d,    pairal^Gl    liri^s  that  extend 


I^o^i^on  ta^lly  dvom  tiim  trailing  edge 


ctE   tS^G  ^objoct:   puir ported  to    be  in 


nno  ^i^n . 


.  .       ^i:^ph  ic  aGvice    usg^  to  irnply 
cr^n^ic  laovQm^nt,  it    consists  at 


sliQr-t  1  iiies  wSiich  ecl%o  n  part  al 


tliQ  ^ig  lire   pu^po^ted    to  be  in 
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Blur. . 


h     t      i  * 


a  grnphic  device  used  to  indicate 


girQ£it  spGGd,   it  conslsts  of  n 


shaded  area  g:< tending  from  the 


purported  to  be  in  motion. 


tteiiltncj  edgQ  of  the  figuru 


Auction  posQ  ,  *  ,  * 


.Six  tter thv>?&i  te  de fines  the  action 


o 


pose  as  an  implied  motion  cue 


.1 1  cons  istfi  of.  a  representation 


thy  object  itself'    (1965,   p.  18). 


which  is  an  integc^al  part  of 


of  a  himan  figurG  in  profile  v;ith 


i  ts  body  and  li'iibs  arrancjed  in  a  manner  deviating  from  a 
strictly  vertical,    standing  position.     For  the  purposes  of 
statisticcil  treatniont  in  this  study,   however,   the  action 
pose  is  regardod  as  an  active  object,   not  as  a  motion  cue 
por  * 

The  objective  of  this  study  was  to  nieasure  motion 
perception  of  subjects  in  three  aga  groups   (four,  seven, 
and  eleven  year  olds)/  corresponding  to  Piaget's  pre^opera- 
tional   through  concrete  operational  poriocls  ,   to  determine 
wdiothor   (a)    tho  implied  itiotion  cues  ar^  perceived,  Kdiether 
(b)    thnre  is  an  hierarchical  order  of  identification  of  the 
SQlccted  cues,   and  \vhother   (c)    the  configuration  of  the 
object  purportod  to  bo  seen  as  moving  affects  the  perception 
of  motion.     The  conf iguration  of  object  type  may  be  said 
to  con  tain  two  dimensions ,   shape  and  activi  ty . 

The  quofstlons,   then,  which  guided  this  study  v;ere: 
1,     Do  children  at  difforcnt  dcviplopraental  levels  attend 
to  implied  motion  cues,   nnd  do  they  ri:£id  than  as  motion  devices? 
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2*     Is  one  scjlacted  impliQd  motion  cue  perceived  mor© 
often  as  containing  motion  inCormation  than  is  another  sel^ctad 
implied  motion  cue  by  children  as  they  advance  frorn  the  pre- 
operational stage  to  the  coiicrete  operations  stage? 

3*     Do  certain  inherent  characteristics  of  the  kind  of 
ob j ects  be ing  acted  upcon  differentially  affect  the  corrLinunicative 
power  of  selected  implied  motion  cues  as  reported  by  children 
as  they  maturG  froni  the  pre--operational  to  the  coiicrete 
operations  stage? 

The  last  of   these  questions  requires  some  aclditional 
consideration.     Both  Satterthwaite,  and  Friedman  and  Stevenson 
{1975)    found  tho  nature  and  nonf iguration  of  the  object 
critical   to  the  question  of  motion  perception  in  non^inotiQn 
media.     Each  study  found  that  active,   postural  figures  were 
full  of  inotion  inforniation  for  all  of  their  subjects,  regardless  . 
of  the  conventional  cues  c^ip^plicd*     Friedman  and  Stevenson 
did  notf   however,   include  geometric  figures  in  their  study. 
Satter thwai te  did,  but  found  no  perceivable  differences  in 
the  spg^^d.  at  active  or  static  geometric  figures,  although 
rounded  objects  were  seen  as  moving  faster  than  angular 
objects  » 

This  Btudy  attenipted  to  examine  all  three  kinds  of 
objects -'--hum  an  figureR   (postural),   angular  geometric,  and 
rounded  geomritric--" to  assess  their  interaction  with  the 
moti,on  cues  selected  for  this  study  and  with  subjects  at 
their  various  stages  of  development*     Additionally,  an 
active  and  static  object  wora  presented  for  each  Kind  of 
object*     The  objects  may  be  classified  as  in  Figure  1* 
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Figure  1.       Classification  of  kinds  of  objects. 


|^Ub_25£MjL  this  study  there  v/ere  3  0  four  year  olds, 

30  seven  year  olds,   and  30  eleven  year  olds,     T'hey  were  selected 
arbitrarily  from  the  National  College  of  Education  Baker 
Demonstration  School^  and  the  Martin  Luther  King^  Jr.^  Labora^ 
tory  School  in  Evans  ton ^  lllinQis,     The  subjects  ware  primarily 
from  niiddle  to  unper-middla  class  farnilies- 

n^te£i_alSj_    A  ssrios  of  24  carcla  served  as  the  stimulus 
materials »     Each  card  contained  on^  illustration  of  one  of  siK 
objects:   circle/  ellipse,   square^  rectancjle,  running  figure^ 
and  standing  figure*     The  objects  were  of  three  kinds: 
human  figures,  angular  geometric  figures,  and  rounded  geo*-- 
metric  figures,     There  were  two  dimensions  of  each  object 
type^  active  and  static.     The  active  homan  object  was  repre-^ 
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sented  by  a  running  figure.     The  static  human  object  v/as 
represented  by  a  standing  figure.     The  active  angular  geomtstric 
object  was  represGnted  by  a  rectangle.     The  static  anguLat 
geometric  object  was  represented  by  a  square*     The  acti^fe 
rounded  geometric  object  was  represerited  by  an  ellipse. 
The  static  rounded  geometric  object  wbb  represented  by  a 
circle.     Three  implied  motion  cues  were  added  to  one  picture 
of  each  object,   and  one  picture  of  each  object  stood  without 
any  implied  motion.^^cu^J  /  £or  a  total  of  24  pictures.  The 
selected  implied  motion  cues  were  the  flow  line,  vibration 
mark,   and  blur.     Figure  2      illustrates  the  24  pidtures. 

Procedures.     First/   the  four  year  olds  v^ere  screened 
to  ascertain  their  grasp  of  the  concept  of  real  movement. 
They  were  asked  to  stand  still?   to  run,  to  jump,   to  move  a 
"little  bit",   to  "shiver  and  shake  like  when  you  are  cold." 
They  were  then  given  a  doll,  a  ball,,   a  rectangular  block, 
a  square  block,   and  an  elliptical  block  and  asked  if  each 
was  moving,   and  could  they  make  the  objects  move*  Two 
subjects  who  did  not  understand  the  concept  o£  real  movement 
were  replaced. 

All  subjects  v;c2re  given  a  classi  Eication  test  which  in- 
volved labeling  each  picture  as  *'mouing"  or  "standing  still/' 
The  eKercise  was  carefully  eKplained  to  each  subject;  he  v^as 
told  that  if  any  picture  seemed  as  though  it  were  moving,  or 
as  though  it  were  supposed  to  be  mov/ing,  even  a  little  bit, 
it  should  be  classified  as  moving.  The  subjects  were  handed 
one  picture  at  a  time-^-a  different  randon  order  was  used  Cor 
each  subject--and  the  eKperimenter   (the  author)   asked  the 
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^    Q    Figura  2.    Illustration  of  picturis  used  in  studv 


childrGn  whether  they  thoucjht  that  the  thing  in  the  picture 
looKecl  like  it  was  supposad  to  be  moving  or  standing  still. 
The  eKperimenter  rQcotded  an  H  on  the  pichurG  if  the  ohject 
was  de:icribed  as  movincj^   or  an  S  on  the  picture  if  the  object 
was  describod  as  standing  still, 

^fter  the  test  was  completed,   the  24  cards  werea  gathered 
and  stapled  in  the  same  sequencQ  aB  presented,  and  the  exper^ 
imanter  interviewed  each  student*     Tim  interview  was  informal, 
and  students  were  askf-id  to  eKplain  the  reasons  for  their 
classifca tions .     The  eKperimen tcr ' s  GXpectations  werG  that 
the  older  children  would  identify  the  graphic  conventions, 
whareas  the  youngest  children  would  probably  pay  more  attention 
to  the  nature  of  the  object.     The  dialog  would  provide  additional 
insights  into  how  children  read  the  pictures.      (Hengen  suggested 
that  extensive  interviewing  he  condiicted  to  facilitate  under-- 
standing  the  thought  processes  involv^Kl  when  a  subject  interprets 
an  illustration   (1970,   pp. 48-49).     It  is  also  the  technique 
used  by  Piage t . ) 

appendix  A  contains  the  raw  data  used  in  this  study.  Each 
ob jc J t/mo tion  cue  picture  identified  as  moving  received  a  score 
of  1,   each  identlfiGd  as  standing  still  scored  £-     An  age 
group  K  sex  a  kind  of  object  K  activity  of  object  K  motion 
cues  analysis  of  variance,  with*  repea^ted  measures  on  the  last 
three  variable,  was  used  to  analyze  and  evaluate  the  data 
(Winer,   1971).     The  eKperimental  design  is  shown  in  Figure  3, 
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Illustration  of  experimental  design , 


RG^su-lii-s  and  Discussion 

The  analysis  of  variance  shov;ed  neitlier  thm  main  effect  of 
seK  nor  its  interactions  with  other  variables  were  significant. 
To  siniplify  the  prGsentation  of  the  results,   a  second  analysis 
was  done  from  which  the  variable  sex  was  excluded*     The  results 
of  the  first  analysis  will  not  be  discussed  ♦     The  suinmary  of 
the  second  analysis  is  shown  in  Table  1*     As  shown  in  Table  1, 
all  main  effects  and  interactions  were  significant. 

The  primary  concern  of  this  study  was  to  see  how  a  child's 
stage  of  developmeiit  is  related  to  his  ability  to  perceive  or 
"read"  conventional  graphic  devices  described  here  as  implied 
motion  cues.     Additionally^   it  was  speculated  that  both  the 
kind  of  object  and  the  object's  inherent  motion  characteristics 
might  be  differentially  related  to  the  manner  in  which  children 
at  different  stages  of  development  interpret  intended  motion 
symbols.     The  four-way  age  group  x  kind  of  object  x  inherent 
activity  of  object  x  implied  motion  cue  interaction/  illustrated 
in  Figure    4,    seems  to  provide  the  best  framework  v;ithin  which 
to  examine  the  outcomes  of  this  study.     All  lower  level  inter^ 
actions  appear  to  support  the  observations  made  based  upon  the 
higher  order  interaction  without  adding  new  insights  and^  there- 
fore, will  not  he  addressed  specifically* 

Because  of  the  complexities  involved  in  relating  the  results 
of  the  four'-way  interaction,   the  discussion  which  follows  will 
be  presented  in  the  follov^ing  manner.     The  human  figure  will 
be  discussed  first  because  of  its  dramatic  contrast  to  the  other 
kinds  of  objects  in  its  active  dimesion,  and  because  of  he 
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Table  1 

SurfUTiary  of  Aruilysiy  of  Varianau 
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55,86* 
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*£  <  .001 
*  *  p  <  .05 
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-^ai^iablG  affGct  ag^  SQf^med  to  have  upon    reading  oS  motion  in 
it&  static  diinensifDn  wlaGn  motion  cues  wer"e  added.  Following 
that,    because  cleve J.oijniental  levGl  and  the  reading    of  motion 
au&B  %^ere  of  pxtma^y  in^terest  to  the  study/  obser^ati&ns  will 

made  about  v/hat  hm  been  learned  about  the  age  group  and 
rnotion  cue  variables  ^  Thern  observ^ations  will  be  made  about 
the  rnore  abstrcict  geCTetric  shapes  and  their  inher-ent  acti^^ity^- 

they  rolate  to  ^n^h  other  as  well  as    toother  irariables. 
Conclusions  yill.        stated   and  sugqestions  for   further  rasea^ch 
v/iLl  bQ  advanccjd* 

Peajing_tlTiD  II  lus  tcrtigy  _o£  the  Four-way  Interaction 

A  guide  to  reading   the  four-way   interaction  is  presentedl 
in  Pigure  5.       The  reader  is  advised  to   examine  tha^t  figure 
before  explDxiiig  the   illustration  of  the    four-w^ay   i  nterrac tion 
in  Pigure  4. 

Beading  the  gcaphs   in    Figure  18  from  left   to  ^  igh t  deingrt- 
stratus   the  different  ways    in  v/hich  the    three  age  g  roups  re- 
ported pRrnedvcd  motion  for  a  giv^en  object  type  and  activity 
staif.     Hi^ading  from  bottom  to  top  show-s    \m  the  addition  of 
a  p£irticular  motioa  cue  may  have  altered    thoso  perceptions, 

H  urn  air^°  F  i  g  u  r  e 

One  dramatic  iresult  was  that  the  active  (running)  hurnan 
figure  Was  almost  aiv^ays  seen  as  movring  ^    regardless  of  either 
a^e  o£  subject  or  niotion  cue  applied.     The  running     figure/  or 
actiori  pose,   cleariy   possesses  inherent  motion  infoOTation  f^r 
ail   age  groups  studie?d, 
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01,  Human  figures 

02,  Angular  geonnetric  figures 
03j  Rounded  geometric  figures 


FLOW  LINES 


Active 
Activity  of 


Static 


object 


GuidG  to  reading  Illustration  of  four-^way 
interaction.     The  graph  in  this  figure  is  the 
same  as  the  graph  in  the  upper  left  corner  of 
Ficjure  4.       It  shows  the  pGrcentage  of  4  year 
olds  who  perceivecl  picturcis  as  moving  when 
flow  linos  wore  addod  to  each  of  three  objects/ 
for  cither  of  tv^o  activity  states.     The  solid 
black  line  shows  that,   in  the  presence  of  flo^ 
lines,   1001  of   the  4  year  olds  saw  the  active 
(running)   human  figure  as  moving ,  while  131 
saw  the  static   (standing)   human  figure  as  moving. 
The  dotted  line  sho%^s  that,   in  the  presence  of 
flow  lines,   10  r>  of  the  4  year  olds  saw  the  active 
(rectangle)   angular  geometric  figure  as  moving, 
while  61  saw  the  static   (square)   angular  geometric 
figure  as  movirig  ,     T^he  dash--line  shows  that,  in 
the  presence  of  flo\v  lines,   20%  of  the  4  year  olds 
saw  the  active   (ellipse)   rounded  geometric  figure 
as  moving,  while  26%  sow  the  static  (circle) 
rounded  geometric  figure  as  moving* 
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The  static    (standing-)   huriia?^  fLgU3^e  clea&rly   presents  guite 
differant  inf^ritiation,     A^ltho^igJ  it  im  slmo^t  c onsistentLy^ 
reported  as  having   less  iTLotiori  iafp^ritishion  than   the  other^ 
static  figures ---square  and  rectsn  gLe^--the  addition  of  motion 
cues  apparentLy  differentially  sf  f^cts  the  a^Joujit  ofmotloji 
perceived  by  children  &t  di  ff  erei^  t  sta^ges  ^     All    three  age  ^ro^ips 
reported  no  motion   f^r  the  static  f  igi^re  under  -the  rio  cue  con^ 
ditlon*     As  atiy  on©  o^f  the  3<inds        cifles  %^as  addedf  howexrer, 
the  two  older  groups   repo^t-ed  that   th^  st^nd  ing    figLjre  W3l& 
"moving*"     The  pxQ^schocl^rfif   on  th  e  oett^er  h  and  ^  reported  almost 
no  motiort. 

This  Latter   result  s^eerns  rea^o  m^le.     i  t  would    be  totally 
inconsistent  witli^ealit^  ajid  most  aHp^e^ienc  «  for  i^ound  children 
to  assom©  thah  a   atancliang  liiunan  f4g%ira-,   feet  tog-ether^  body 
rigid^  was  rnoving,     Ths  pa^e^ence        ab^stract  motiori  cues 
apparentLy  did  not  o\/er:ride  '*no^in^t  ion^-   sign^^ls    inherent  in 
the  standing  Jigare.     rlie  fact  th%t  in.^  most  ^ases  hhaa  older 
Qhiidren  reported  less  nioti^ri  for  tJe    static  human  ffigure  than 
for  thm  static  geometries   (^1  though  th>G  statistical  significance 
of  this  vas  not  tested)   may  be  sn  iJidi  cation  that  e\ren  th^se 
children  v/ere  a  &it  relxictarit  to  id-^nt  iffy  a   "standir%g"  figure 
as  -'movirtg *  " 

Ag e  Group  and  jjofcipji  Cp.m 

In  generA-lf    the   fo^jr-v/^y  int^r^ct  i^n  %Q^m    to  ^eflec-t  the 
fact  that  v/ith  ir^ct^ easing  a^e  the  \r^ri  qhs  mo^tiori  cu^i  had  an 
increasingly  heterp^eaieous  effect  o^^  t  \m  pertf^eptiori    of  motion 
as  the  motiori  cu^s  addgd  vvG:te^  c'e^p^ct  i^ely^    blurs,  vibration 
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marks,  and  Elov;  linGS.     tHq  percentage  of  subjects  perceiving 
the  five  objects   (oLher  than  the  running  figure)   as  moving  in 
the  graphs  in  Figure    4    for  the  seven  and  eleven  year  olds 
increases  \^ith  restrict  to  that  same  order  of  addition  of  motion 
cues,  while  the  fopr  year  olds'   graphs  remain  essentially  un- ■ 
changad  from  the  no  cue  condition.     An  explanation  consistent 
with  this  observation  v/ouid  be  that  the  older  children  v;ere 
"reading'*  the  cues^  while  the  prQ-schoolars  %mr%  ignoring  theni , 
centering  their  attention  on  the  object  depicted^ 

In  interview  conrlucted  by  the  author  with  the  students 
following  the  aclin  ini  stration  of  the  experiTnent,   the  older 
groups  frequently  explained  that  things  seemed  to  be  moving 
because  of  "the  lines'*,  or  motion  cues.     Only  one  four  year 
old  identified  the  lines  at  all*     That  the  older  children 
were  probably  ''teacling"  tlie  cues  is  also  suggested  by  the 
differential  effect  the  vfirious  cues  apparently  had*     For  the 
seven  and  eleveti  year  olds  flov/  lines  and  vibration  mark.s  seeined 
to  be  more  effective  than  the  blur  motion  cue  for  most  objects. 
This  is  plausible  because  the  Interviev/s  revealed  that  the 
blur  was  probably  a  rnore  ambigupus  cue  than  the  other  graphic 
conventions  »     It  received  the  widest  ranije  of  i n torpretations  # 
several  inconsistent  with  motion  interpretation*     Some  sav;  it 
appropriately  as  a  "bunch  of  little  lines  close  together, 
really  moving  fast."     But  others  saw  the  blur  as  a  Hornado'S 
or  ''air  pushing'S  or  merely  as  -'a  %vall",  or  "a  shadov;." 
Perhaps  in  the  absence  of  contextual  clues  the  blur  motion  cue 
as  drawn  for  this  study  was  a  more  confusing  sign  than  were  the 
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other  two  graphic  cugs.     tHq  fact/   hov/ever,  that  even  a  poorly 
drawn  Que  was  retJortQcl  on  the  average  morQ  than  SO  per  cent  of 
the  time  as  contributing  to  motion  interpretatiori  may  hm  further 
evidence  that  pictorial  literacy  has  advanced  enough  by  th©  age 
of  seven  to  dif f eren tiate  the  older  groups  from  the  nursery 
school  children. 

Qbj ect  and  Ac tivlty 


Tha  pov;or  of  the  running  Imman  figurQ  as  a  rnotion  indicator 
has  already  beon  diKcusstid.     A  clo.sGr  look  at  thc^  perc^ntagf?  of 
subjects  perceiving  movoment  in  the  georaetric  figures  suggests 
that  the  variance  between  the  tv70  kinds  o£  geometric  shapes 
could  Idb  chanc  variation.     In  all  but  one  inytancQ   (see  the  top 
center  graph  in  Figure  4)      rounded  geomotrics  v;ere  perce^ivcd 
as  moving  inore  than  emgul^r  geometries,   regardlGSS  pf  activity 
state  inherant  in  the  objocts;   the  almost  horizontal  slopes  and 
sometimes  nearly  paralial  nature  of  the  lines  in  Figure  4  sug^ 
gests  that  differences  between  classes  of  geometric  objects 
were  not  very  meaningful. 

Similarly  f   the  pc^rcentagG  of  aubiacts  perceiving  niovement 
in  the  geoniGtric  figures  suggQSts  that  the  variance  v/ithin  the 
activity  dimension  could  also  he  chance  variation  in  most  cases* 
This  is  alniost  surely  the  case  regarding  the  active  figures^^ 
ellipse  and  t^ec tanglo-'--and  usually  the  case  regarding  the  static 
f igures-^^circle  and  square.     It  is  possible,   however,   that  undei^ 
certain  conditions   (see,   for  eKample,   the  seven  year  old/vibra^ 
tion  mark  graph  in  Figure    4)    the  circle  may  be  a  better  motion 
indicator  than  the  squarG/  al bei t  the  statistical  significance 
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of  this  ^'as  not  tested* 

In  iritervlei^s  conducted  by  the  e3<periinenter  several  of 
the  four  year  oids  expressed  some  ideas  about  the  geornetric 
objects  ^hich  Way  e:<plain  some  of  the  differences  betv/een 
subjects'  reGt)onsos  to  geometric  ob j ecfcs--^both  active  and 
static^-'and  to  the  active  hurtian  figure*     Corninents  like  "a 
square  oan*  t  move"  v/ere  very  common  among  the  pre-schoolers. 
They  also  said  that  ''the  man  is  running**  or  "the  mari  is  jamp- 
ing,"     This  niay  indicatQ  that  as  four  year  olds  look  at  the 
geometric  shapes,  perhaps  some  intuitive  idaas  about  friction 
or  inertia  are  operating. 

Conclusions 

The  results  of  this  study  seem  consi stent  with  the  findings 
of  Friedman  and  Stevenson   (1975)*     In  both  studies,  younger  and 
older  subjects  did  not  classify  pictures  in  the  same  manner. 
Reliance  on  Impliad  motion  cues  apparently  does  increase  \v^ith 
age » 

The  current  study,  using  ssven  year  olds  rather  than  six 
year  olds,   seems  to  establish  rnor©  precisely  the  timt^  period 
when  children  may  begin  to  '*read"  motion  cues.     Friedman  and 
Stevenson  founci  that  four  and  six  year  olds-^-both  vi  thin  the 
pre-operationaL  stage-^^saw  mos6  postural  pictures  am  '^moving'' 
and  that  they  did  not  see  any  difference  in  the  way  graphic 
devices  affected  motion  perception*     Sixth  graders  and  college 
students  classified  approKimately  the  same  number  of  postural 
pictures  with  or  without  motion  cues- as  ''maving"   (p*  777).  The 
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current  study  found  the  behavior  of  seven  and  eleven  year  olds-- 
at  the  beginning  and  end  of  the  concrete  operations  stage--to 
be  more  like  each  other  than  like  the  behavior  of  the  pre- 
operational four  year  olds.     It  mmy  be  that  by  the  time  young- 
sters  enter  the  concrete  operations  period  they  have  the  ability 
to  "read"  conventional  motion  cues,  but  not  much  before*  The 
current  study  used  geometric  shapes  as  v;ell  as  postural  figures; 
Friedman  and  Stevenson  used  only  the  latter,     That  the  older 
children  seemed  to  attribute  a  substantial  amount  of  motion  to 
both  kinds  of  objects  when  motion  cues  ^s^ere  used  may  support 
and  emphasize  Friedman  and  Stevensori's  sugcrestion  that  once 
implied  motion  cues  are  learned,  they  assuine  the  same  poxver  in 
eliciting  a  reading  of  movement  as  do  action  pose  kinds  of 
cues,  , 

ComparisQns  with  the  Satterthw^ite  study   (1965)  must  be 
more  conservative.     Satterthwai te  was  concerned  with  the  quality 
and  degree   (i,e*,   smoothness,   speed  and  distance)   of  motion  as 
reported  by  college  students.     Although  sonie  similarities  seeni 
ta  exist  between  the  s tudies^-^rounded  objects  were  generally 
seen  more  often  as  moving  than  were  angular  ones,  active  and 
static  geometric  figures  seemed  to  hau©  similar  amounts  of 
motion  information,  and  the  running  figure  was  always  more 
potent  than  the  standing  figure---- tests  for  simple  effects  were 
beyond  the  scope  of  either  stud;/. 

The  facts  which  do  seem  to  emerge  clearly  from  this  study 
are  that  the  active  human  postural  figure  was  understood  by  the 
children  in  this  study,  regardless  of  their  age  group,  as  moving 
with  or  without  the  addition  of  motion  cues,  and  that  once  the 


children  left  the  prc-o pera tional  stage  of  development  they 
apparently  had  the  ability  to  "read"   the  implied  motion  cues 
used  in  this  study.     The  implication  for  niessage  designers  in 
general  and  for  those  interested  in  illustrating  books  for 
young  children  in  particular  seems  cleat,     The  differences 
between  children's  ability  to  undGrstancI  implied  motion  cues 
at  their  respective  stages  of  development  ought  to  be  taken 
into  account  when  prodiicing  piatorial  rn^terials. 

SuggGStions  for  Fur tjiej.^  Research 

Other  selQCtGd       tion  _cu^eg^    Only  three  iitiplied  inotion 
cues  v/ere  SGloctcd  for  this  study ^  and  one  of  them" the  blur---- 
v/as  apparently  more  ambiguous  than  the  otliar  cueS/  perhaps  be- 
cause it  was  pQorly  clrav^n*     A  study  using  additional  motion 
cues^  or  variations  of  the  cues  used  in  the  presGnt  study 
could  be  developed*     The  relative  powex^'  of  different  cues  to 
elicit  motion  interpretations   Erom  pictorially  literate  subjects 
could  be  explored  . 

C o n t e K t u a l_c  1  Up a  .     All  of  the  pictures  usc3d  in  the  current 
study  were  essQntially  contaKt-free  and  in  that  sense  were  un- 
realistic*    It  would  be  interesting  to  further  test  Stevenson 
and  Friedman -s  hypo  thesis-- that  pictorial  cues  with  structures 
very  similar  to  structures  in  the  thrQe-difnensional  world  v;ould 
be  easy  to  intGrpretf  bat  that  pictorial  cues  less  than  iso- 
morphic with  reality  would  be  more  difficult  to  interpret   (p.   773) — 
using  more  lifelike  pictures  and  settings  than  were  used  in  the 
current  study.     For  eKample,  FigurQ    6    sHqws  a  boy  on  a  bicycle- 
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Illustration  of  Robert  HcCloskey-s 
cartoon  technique.     Note  the  artist's 
use  of  flov/  lineB*     Fruiii  Haka  Vlay  fo^ 
Ducklings /  by  Bobert  McCloskey,  copyright  (c) 
1948/    By  permission  of  the  Viking  Press ^  Inc. 


Most  readers  would  agree  that  he  and  his  bicycle  are  supposed 
to  be  in  motion.     The  illustrator,  Robert  McCloskey ,  has  pro- 
vided many  motion  cues.     Although  the  cartoon  technique,  in- 
cluding the  use  of  flow  lines,   is  an  abstraction  of  reality, 
there  are  enough  lifeliKe  cues  such  as  the  boy's  position  on 
the  bicycle,  his  tie,  shirt-tail,  and  hair  flying  to  assure 
that  most  readers  would  acknowledge  that  the  boy  and  the  bi- 
cycle are  "moving."    One  could,  however,  manipulate  this  draw- 
ing, maintaining  some  contextual  clues  while  eliminating  others. 
t'7ouia  some  readers,  for  examplR,  think  that  the  boy  and  his 
bicycle   (in  Figure  14}   were  moving  if  the  boy  were  drawn  sitting 
erect,  his  back  perpendicular  to  the  ground,  his  feet  on  the 
pedals?    l7ould  this  be  the  contextual  equivalent  of  the  standing 
figure  in  the  current  study? 

In  the  current  study,   in  the  absence  of  contextual  clues, 
it  appeared  that  pre-schoolers  ignored,   for  eKainple,   the  flow 
line  motion  cue.     Would  a  young  child  be  more  likely  to  "read" 
a  flow  line  if  it  helped  him  to  interpret  the  picture  within 
a  specific  context?     Imagine  a  drawing  of  a  boy  in  the  process 
of  cliinbing  a  flagpole.     Might  the  placement  of  flow  lines  above  ' 
or  below  the  boy  be  used  to  indicate  if  he  was  climbing  up  or 
down  the  pole,  and  would  young  children  use  such  cues  to  decide 
the  direction  of  motion?     Experiments  qould  be  designGd  to  explore 
these  and  similar  questions. 

Satterthwaite  follow-up.     Satter thwaite  explored  several 
ideas  which  the  current  study  did  not.     It  would  be  interesting 
to  submit  to  a  more  rigorous  statistical  treatment  questions 
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about  perceived  tense   (e.g*,   in  the  presence  of  different  cues^ 
or  of  cues  of  greater  length  or  thickness/  does  an  ob^ject  appear 
to  be  about  to  starts  already  in  motion/  or  about  to  stop?); 
speed   (e,g,,  does  an  object  in  the  presence  of  one  implied 
motion  cue  appear  to  be  moving  faster  than  the  same  object  in 
the  presence  of  another  implied  motion  cue?) ;  or  quality  of 
motion   (e.g.  /  v/hich  inotion  cues  are  best  used  to  indicate  erratic 
motion;   fluid  motion?). 

Survey  of  artists.     It  would  be  interesting  to  survey  artists 
to  find  out  what  kinds  of  assumptions/   if  any ^   they  make  about 
children's  pictorial  literacy/  and  to  ascertain  what  techniques 
they  would  use  under  what  conditions  to  imply  motion.     They  might 
be  asked  if  they  attempt  to  adapt  their  techniques  to  various 
audiences /  or  if  they  would  modify  their  style  if  it  v;ere  demon-- 
strated  that  groups  of  readers  responded  in  different  ways  to 
different  styles,  h  survey  instrument  could  be  designed  to  learn 
more  about  the  community  of  children's  book  illustrators,  and 
these  and  other  questions  could  bo  asked  to  loarn  more  about 
the  concerns  and  attitudes  of  this  group  of  "message  designers," 
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This  study  reports  an  inv^estigntion  of  how  children  at 
different  stages  of  developnien t  understand  implied  motion  cues. 
The  function  of  these  cues  is  to  represent  motion  in  non^motion 
media   (i,e,,   children's  books,   textbooks^  magazines,  and  news- 
papers) .     The  implied  motion  cues  selected  for  this  study  \^ere 
the  flov/  line,   the  vibration  mark,  and  the  blur* 
Problem 

Prior  rGsearch  was  GKamined  which  had  shown  that  visually 
encoded  messayes  communicate  in  a  manner  similar  to  verbally 
encoded  ones.     How  well  a  person  understands  the  significant 
verbal  symbols  of  a  language  and  their  temporal  and  spatial 
arrangement  is  a  measure  of  one's  verbal  literacy;  how  well 
a  person  understands  a  language  composed  of  visual  symbols  and 
their  spatial  arrangement  can  be  said  to  be  an  index  of  one's 
visual  literacy*     The  present  study  was  interested  in  v/hat 
might  be  called  pictorial  literacy  and  more  specif ically ,  with 
respect  to  a  particular  element  of  pictorial  aommunication , 
namely /   implied  motion  cues. 

People  are  not  born  literate.     Through  a  combination  of 
factors';   including  development,  general  experience^  and  formal 
education,   people  eventually  learn  to  decode  written  and  drawn 
messages.     Extrapolating  from  Piaget's  theory  of  cognitive 
development  *   it  was  suggested  that  a  parallel  exists  between 
the  development  of  spoken  language-symbol  systems  and  the 
development  of  other^symbol  systemic  specifically^  pictorial 


such  .3  flov  linss         Vibration  .a..s  ..0.  ,y  „tists  to 

depict  motion  would  have  little  nr  ^ 

±±^z±m  01  no  meaning  to  pre- 
operational   (DrQ'-qf-hnnI  1  1  J 

iFic  facnooij   children,   but  fh^+- 

^ii/   Duc  tnat  an  action  pose 

^eing  „o«  li.e  „al  h™a„  pos.u.e,  ,,o„ia  be  a  s.„bol  witMn 
the  grasp  oi  understandin,  of  chiaaren  «thi„  this  ,„„p.  By 
the  .eginnin,  of  Piagefs  ooncrete  operation,  sta.e,  a.out  " 
seven  .ears  of  aoe,  .oat  children  whouia  be  capable  of  onder- 
sun.in,^  arbitrary  graphic  si.ns,  and  by  age  eleven  virtually 
a^l  shuuiu  be  capable. 

.      It  vas  further  sus^est.d  that  the  Mnd  of  object  .pen 

0^  -venent.     „^a.  figures,  for  e.a^pie,  are  .ore  ii.eiy  to  ^ 
appear  to  be  in  .ot.on  than  are  geonetrio  shapes  such  as 
squares  or  circles.     it  was  speculated  that  an  inberent  .otion 
'actor  .ay  also  bo  said  to  reside  .ithin  an  object,  such  that 
-  appears  to  be  ^oro  active  or  „ore  static-e.,,,  runnin, 
^i^ures,  ellipses  and  rectangles  .ay  be  seen  as  .ore  aotive 
than  standing  figures,  circles  or  squares. 
Three  questions  were  posed. 

1.     Do  children  as  they  grow  older  attend  to  flow  lines 


devices? 


2.     IS  one  selected  implied  motion  cue  perceived  more 
Often  as  containing  motion  information  than  is  another  3el.ct3d 
implied  motion  cue  by  children  as  they  advance  fro.  the  pre- 
operational stage  to  the  concrete  operations  stage' 
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3.     Do  certain  inherent  characteristics  of  the  Kind  of 
objects  being  acted  upon  differentially  affect  the  coinmunicative 
pover  of  selected  implied  motion  cues  as  repoirted  by  children 
as  they  mature  from  the  pre=operational  to  the  concrete  operations 
stage? 

Methodology 

Subjects.     Ninety  primarily  middle  class  subjects  — 30 
four  year  olds,   seven  year  olds,   and  eleven  year  olds-=were 
selected  arbitrarily  from  schools  in  Evanston,  tUinois. 

-  - ^„  .     ....  lyAi^sJ.  iiuwiiurtJ.  scuuy  was  conciuctecl  in  an 
attempt  to  determine  how  children  at  different  stages  of 
development  perceived  implied  motion  cues  in  combination  with 
a  variety  of  objects.     Each  subject  warn  shown  the  same  24 
pictures  which  were  arranged  in  different  random  orders  for  each 
child.     There  were  three  kinds  of  ob j ects^-hunian  figures, 
angular  geometric,  and  rounded  geometric-^each  with  an  inherent 
active  and  static  dimension.     Each  of  these  pictures  appeared 
with  one  of  three  mution  cues  and  alone.     The  number  of  each 
of  the  picture  types  identified  as  moving  by  each  subject  was 
determined.     An  age  group  x  kind  of  object  x  activity  of  object 
X  implied  motion  cue  repeated  measuros  analysis  of  variance 
was  used  to  evaluate  the  data. 
Results  and  Conclusion 

All  main  effects  and  all  interactions  were  significant. 
The  four-way  interaction,  F(12,   522)   =  3.07,  p     .001,  was 
illustrated  and  used  as  the  basis  for  presenting  the  results. 
The  data  seemed  to  show  clearly  that  the  active  human 
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postural  figuro  was  understood  by  all  children  in  the  study 
as  "moving''  with  or  v;ithout  tho  presence  of  motion  cues.  The 
pre-operational  children  appeared  to  attend  to  only  the  kind 
of  object,   the  action  pose  being  the  only  object  type^  for 
which  significant  movement  waa  reported.     Motion  cues  seemed 
to  have  no  effect  for  this  group.     The  older  groups  both 
reported  apparently  similar  motion  increases  for  all  object 
types  and  activity  states  when  motion  cues  were  added^-except 
that  the  active  human  figure  remained  consistently  high.  It 
was  not  determined  whether  any  motion  cue  was  significantly 
more  effective  than  any  other/   although  the  blur  appeared  to 
be  a  somewhat  weaker  motion  signifier  in  this  study*  This 
result  may  have  been  due  to  the  blur's  being  rather  poorly  drawn. 

It  was  concluded  that  once  children  leave  the  pre-- 
operational  stage  they  apparently  have  the  ability  to  "read" 
the  implied  motion  cues  used  in  this  study.     The  implication 
for  message  designers  is  that  this  differential  ability 
ought  to  be  taken  into  account  when  producing  pictorial 
materials   for  children. 
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ABSTRACT 

The  study  was  designed  to  investigate  the  effects  of  cartoon  illustrations 
on  fe^le  and  ™ale  college  student  comprehension  and  evaluation  of  information, 
sutjects  were  assigned  to  one  of  five  treatment  groups  as  follows:    (1)  Printed 
text   (2)  printed  teKt  with  cartoons.  (3)  audio-visual  presentation,  (4)  audio 
only  presentation,  and  (5)  visual  only  presentation.    Following  the 
informtive  presentation,  each  subject  completed  a  25-1tem  comprehension 
test  and  measures  of  sel f -reported  enjoyment  and  helpfulness.    The  results 
indicated  that   there  were  no  significant  differences  between  the  printed 
text,  the  cartoon  text,  and  the  audio-visual  presentation  In  terms  or 
comprehension  scores.    Th.  crtoon  text  was  perceived  as  significantly  ™re 
enjoyable  than  any  other  presentation  mode.    There  was  a  significant 
interaction  between  sex  and  presentation  mode  with  males  scoring  significantly 
lower   on  comprehension  in  the  cartoon  text  condition. 


Effect  of  Cartoon  niustratlons  on  the  Comprihension 
and  Evaluation  of  Information  Presented  in  the 
Print  and  Audio  Mode 

The  use  of  visual  humor  In  the  form  of  cartoon  11  lustrations  is  a 
familiar  characteristic  1n  many  college  textbooks  (see  for  example 
Hance,  Ralph,  &  Wiksell,  1962;  Re1n.  1972;  and  Larson,  1979),  and  the  use 
of  cartoon  humor  as  an  instructional  technique  in  the  classroom  has  also 
been  given  considerable  attention  (see  Adams,  1974;  Brooke,  1973-74; 
Neiei  1973;  and  Miner,  1969).    The  focus  of  this  attention,  however,  has 
usually  been  motivational  in  its  orientation  with  little  evidence  advanced 
to  support  any  positive  effects  on  learning  as  a  result  of  the  use  of  the 
cartoon  humor.    Studies  have,  in  fact^  found  that  the  use  of  humor  1n  most 
contexts  does  not  result  in  Information  gains  (Gruneri  1976,  pp.  301-304).  " 

In  a  recent  discussion  of  cartoon  humor  in  magazines,  John  Peter  (1978), 
a  publications  consultant  and  past  art  director  for  McCall 's,  advanced  four 
arguments  for  the  use  of  cartoons'    (1)  high  readership,  (2)  enjoyment, 
(3)  increases  the  visual  element,  and  (4)  provides  an  identity  for  the 
publication*    In  his  discussion  of  these  advantages,  Peters  makes  specific 
reference  to  the  effect  of  cartoons  on  reader  abilities  to  remember  the 
content  of  cartoons,  thus  suggesting  a  learning  effect. 

The  present  study  was  designed  to  Investigate  the  effect  of  cartoon 
illustrations  on  the  comprehension  and  evaluation  of  information  presented 
in  combinations  of  the  print,  audio,  and  visual  mode. 

Though  there  has  been  a  significant  amount  of  research  investigating  the 
effect  of  presentation  mode  on  learning,  no  clear  conclusions  can  be  stated 
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The  effect  of  the  sex  of  the  learner  has  been  almost  totally  naglected 
in  studies  of  the  relationship  between  presentation  mode  and  learning.  In 
the  one  study  which  used  high  school  or  adult  subjects  and  focused  on  ::he 
effect  of  %ex,  Dwyer  (1971)  found  no  differences  in  learning  from  several 
types  of  visual  presentations. 

The  present  study  attempted  to  investigate    (1)  the  effect  on  the 
comprehension  and  evaluation  of  information  presented  in  several  combinations 
of  print,  audio,  and  visual  formats,    (2)  the  effect  of  the  aex  of  the  learner 
on  the  comprehension  and  evaluation  of  information  presented  1n  several 
combinations  of  print,  audio,  and  visual  formats,  and  (3)  any  interactions 
between  pfesentation  mode  and  sex  of  the  learner. 

METHOD 

Subjects.    Subjects   were  150  student  volunteers  enrolled  In  several 
sections  of  basic  communications  classes  as  Virginia  Polytechnic  Institute 
and  State  University.    Half  the  subjects  were  feniale,  and  half  were  male. 
While  no  coursework  credit  was  given  for  participation  in  the  study,  the 
students  were  told  that  if  their  final  grade  average  In  the  course  was 
borderline,  they  would  be  given  the  higher  grade  in  the  final  averaging. 

Materials.    The  materials  developed  for  use  in  this  study  were  (1)  a 
basic  text,  (2)  an  audio  recording  of  the  basic  text,  (3)  cartoons  based  on 
the  basic  text,  and  (4)  slides  of  the  cartoons. 

A  basic  text,  "The  Library  at  Virginia  Tech,"  was  developed  from 
information  available  in  the  library  as  well  as  with  the  aid  of  several 
members  of  the  library  staff.    The  text  described  four  keys  helpful  in 
unlocking  the  resources  of  the  library. 

An  audio  cassette  recording  was  made  of  a  female  reading  the  basic  text. 
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was  made  to  provide  vocal  variety  in  the  recorded  version  other 
normally  associated  with  a  person  reading  informative  material 

ies  of  forty-three  cartoons  was  created  and  drawn  by  a  student 
oons  appeared  1n  each  issue  of  the  campus  newspaper.    The  cartoons, 
ured  characters  from  one  of  his  regular  strips,  were  created 
ly  for  this  study  to  accompany  the  content  of  the  basic  text, 
-and-white  slides  were  made  from  each  of  the  forty-three  cartoons, 
ments.    Subjects  received  one  of  five  experimental  treatments.  The 
tment  was  a  simple  printed  copy  of  the  basic  text.    The  booklet 
e  a  handout  one  might  receive  1n  a  class  or  upon  entering  the 
ee  Appendix   c   for  some  sample  pages)*    The  booklet  was  seven 
ced  pages,  and  there  were  no  visuals  or  headings  to  provide  visual 
changes  in  the  subject  content, 

econd  treatment  was  a  booklet  which  combined  the  basic  text  with 
three  cartoons  (see  Appendix   C    for  some  sample  pages).  This 
s  eighteen  pages  in  length, 

hird  treatment  was  an  audio-visual  presentation  which  consisted  of 
recording  of  the  text  cued  to  the  slides  of  the  cartoons*    The  total 
on  was  presented  by  means  of  a  slide  projector  connected  to  a 
ecorder , 

ourth  treatment  was  the  audio  recording  alone,  and  the  fifth 
was  the  slide  presentation  alone, 

dent  Variables,    There  were  three  dependent  variables  In  the  study, 
multiple-choice  comprehension  test  over  the  contents  of  the  basic 
eveloped  to  measure  subject  comprehension. 


Ten  bipolar  adjective  pairs  were  used  to  measure  affective  responses  to 
the  presehtatlon.    Factor  analysis  suggested  that  there  were  two  well  defined 
factors  which  were  then  used  as  separate  affective  measures.    One  factor, 
called  "Enjoynient,"  contained  six  adjectivi  pairs  {exciting/boring,  interesting/ 
dull,  humorous/serious,  enjoyable/unenjoyabl e,  happy/ sad,  and  unusual/ordinary). 
The  second  factor,  "Helpfulness,"  contained  three  adjective  pairs  (helpful/ 
unhelpful,  informative/uninformatlve,  and  worthwhile/worthless). 

Procedures.    Subjects  were  randomly  assigned  to  treatments  with  equal 
female  and  male  representation  in  Rach  treatment.    The  presentations  were 
in  the  evening  with  order  of  presentation  being  randomly  determined. 

Subjects  were  told  that  they  were  to  read/watch/listen  to  an  informative 
presentation  on  the  library,  and  after  the  presentation  they  would  be  tested 
over  the  presentation.    Time  was  controlled  so  that  all  subjects  were  given 
equal  time  between  the  beginning  of  the  presentation  and  the  time  at  which 
the  test  was  administered.    Since  those  subjects  reading  the  printed  texts 
worked  at  their  own  pace,  they  were  instructed  to  close  their  booklets  when 
they  had  read  the  complete  text  once. 

Statistical  Analysis.    A  2  x  6  multivariate  analysis  of  variance  was 
performed  with  two  levels  of  sex  and  five  treatment  levels;  comprehension 
scores,  self-reported  "Enjoyment,"  and  self-reported  "Helpfulness"  were  the 
dependent  variables.    Following  significant  multivariate  tests,  univariate 
F_  ratios  were  calculated,  and  following  significant  univariate  tests, 
multiple  comparisons  using  the  Scheffe  method  were  computed. 

The  .05  level  of  significance  was  required  for  all  tests,  and  the 
significance  level  is  reported  with  each  separate  analysis. 

The  statistical  analyses  were  performed  using  programs  from  both 
SAS  (Barr,  et.  al.,  1976)  and  SPSS  (Nie,  et.  a]..,  1975). 
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RESULTS 

The  rel lability  of  the  comprehension  test,  deteritiined  by  the  KR-20 
formula,  was  found  to  be  0.78. 

The  results  of  the  factor  analysis  discussed  earlier  is  presented  in 
Appendix  A.    One  of  the  ten  bipolar  items  did  not  fall  into  either  factor, 
and  was  thus  not  used  in  any  further  statistical  analysis. 

The  intercprrelatlons  of  the  three  dependent  measures  are  presented  in 
Table  1.    Intercorrelations  between  Comprehension  and  Enjoyment,  and  between 
Fninv/m«.ni-  and  Hplnfiilness  Were  sioniflcant.    ComDrehenslon  and  Helpfulness 
were  not  significantly  related. 

Table  1 

Intercorrelations  of  Dependent  Measures 

Measure  Comprehension  Enjoyment  Helpfulness 
Comprehension  1.000 

Enjoyment                0.276*  1.000 

Helpfulness              0.086  0.338*  l.OOO 

*£<.01 

Analysis  of  Variance.    The  multivariate  analysis  of  variance  was  used 
to  determine  the  main  effects  of  presentation  mode  and  sex  on  the  three 
dependent  measures.    Univariate  analysis  of  variance  was  used  when  the 
multivariate  analysis  was  significant,  and  when  the  univariate  analsysis 
was  significant,  post  hoc  analysis  of  palrwise  comparisons  was  perfonried 
using  Scheffe. 

The  raw  cell  means  on  the  three  dependent  measures  are  presented  in 
Appendix  B. 


The  main  efftct  of  sex  was  not  significant*    The  main  effect  of 
presentation  mode  was  significantt  multivariate  £(125365)-14*94,  £<.Q01. 
Univariate  £  ratios  were  computed. to  assess  th^  individual  significance  of 
the  three  dependent  measures.    All  three  of  the  univariate  £  ratios  were 
significant  at  £<.O01:    for  Comprehension,  £(4,14g)=28,27^  for  Enjoyment* 
£(4,149)^10,88;  for  Helpfulness,  £(4,149)^6*31. 

The  sax  x  presantation  mode  interaction  was  significant.  The 
multivariate  overall  effect  was  £(12,365)-2.18,  £<,01,    Only  one  of  the 
univariate  £  ratios,  for  Comprehension,  £(4J49)M.49,  was  significant  at 

£<*01  . 

Post  Hoc  Analysis.    The  differences  among  means  for  the  two  dependent 
variables  with  significant  differences  as  computed  using  Scheffe  are 
presented  in  Tables  2  and  3* 

The  differences  between  the  printed  text,  cartoon  printed  text,  and 
audio-visual  presantation  were  not  significant  for  Comprehension.    The  printed 
text  and  cartoon  printed  text  means  were  significantly  greater  than  the  means 
for  the  audio  and  the  visual  presentations,  and  the  means  for  the  audio-visual 
and  audio  presentations  were  significantly  greater  than  the  mean  for  the 
visual  presentation. 

On  the  Helpfulness  measure,  the  differences  between  printed  text,  cartoon 
printed  text,  audio-visual  presentation,  and  audio  presentation  were  not 
significantly  different.    Self-reported  Helpfulness  was  significantly  lower 
on  the  visual  presentation  than  on  any  other  treatment  condition. 

While  there  were  no  significant  differences  on  the  Enjoyment  measure 
when  using  the  Scheffe  procedure,  there  were  differences  when  using  the 
Tukey  LSD  procedure.    These  differences  are  reported  in  Table  4.    The  cartoon 
printed  text  was  perceived  as  significantly  more  enjoyable  than  the  printed 
text,  the  audio  presentation,  and  the  visual  presentation. 


Table  2 


Differences  Among  Mians  on  Comprehension 
Scheffe  Procedure 


Printed 
Text 

Cartoon 
Text 

Audio- 
Visual 

Audio 
Only 

Visual 
Only 

Printed 
Text 

87.9 

2.6 

7.9 

11.9** 

24.7** 

Cartoon 
Text 

85.3 

5.3 

9.3* 

22.1** 

Audio- 
Visual 

80,0 

4.0 

16.8** 

Audio 
Only 

76.0 

12.8** 

Visual 
Only 

63.2 

*£<.05 

**£<.01 
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Table  2 

Differences  Among  Means  on  Helpfulness 
Scheffe  Procedure 


.  *£<.05 

**£_<.  01 
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Text 

Visual 
Only 

Audio 
Only 

11.3 

0.8 

0.8 
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0.1 

2.1* 

Audio- 
visual 

10.5 

0.1 

2.1* 

Printed 
Text 

10.4 

2.0* 

Visual 
Only 

8.4 

Following  the  significant  Sex  x  Presentation  Mode  interactionj  the 
t^-test  was  used  for  individual  comparisons  between  the  means.    The  nature 
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of  the  interaction  is  ploted  in  Figure  1.    On  comprehension,  females  (X=91.467) 
scored  significantly  higher  than  males  (X=79.200)  1n  the  Cartoon  Text 
treatment. (tf 3.11 ,  df*28,  £<.01).    In  one  other  treatment  condition,  Visual 
Only,  male  comprthtnsion  scores  (1^66.667)  were  significantly  higher  than 
female  comprehension  scores  (f=S9.733)  with  t=-2.09,  df=28,  £<.05. 


Table  4 

Differences  Among  Means  on  Enjoyment 
Tukey  LSD  Procedure 


Cartoon 
Text 

Audio- 
Visual 

Printed 
Text 

Visual 
Only 

Audio 
Only 

Cartoon 
Text 

9.3 

1-3 

1.5* 

1.8** 

4.3** 

Audio- 
Visual 

8.0 

0.2 

0.5 

3.0 

Printed 
Text 

7.8 

0.3 

2.8 

Visual 
Only 

7.5 

2.5 

Audio 
Only 

5.0 

*£<.05 

**£<.01 


DISCUSSION 

Of  primary  importance  was  tue  finding  that  there  were  no  significant 
differences  between  the  printed  text,  the  cartoon  text,  and  the  audio-visual 
presentation  in  terms  of  the  comprehension  of  the  content.    This  result 
would  seem  to  lend  support  to  studies  such  as  Van  Mondfrans  &  Travers  (1964) 
and  Main  &  Griffiths  (1977)  which  have  found  that  information  can  be  just 
as  easily  processed  in  one  of  several  sensory  modalities  or  combination 
of  modal ities . 


Additionally,  the  two  printed  formts  (without  and  with  cartoons)  - 
resulted  in  significantly  better  comprehension  than  either  the  audio  or 
the  visual  presentation,  and  the  visual  presentation  resulted  in  significantly 
less  comprehension  than  any  other  presentation  mode.    The  Inability  of  the 
visual  presentation  should  come  as  no  surprise  since  it  was  lacking  in  the 
95% 
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Figure  1 

Sex  X  Presentation  Mode  Interaction 
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essential  information  available  In  the  other  four  presentation  modes.  It 
served  mainly  as  a  control  group, 

A  lack  of  any  relationship  between  the  perceived  helpfulness  of  a 
presentation  and  actual  comprehension  was  demonstrated  by  the  lack  of  any 
significant  differences  on  self-reported  Helpfulness  in  any  condition  but 
the  visual  presentation. 
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The  cartoon  text  was  percaived  as  significantly  more  enjoyable  than 
any  other  presentation  mode.    This  result  does  not  support  Freisinger's  (1976) 
finding  that  the  addition  of  a  cartoon  embellishment  resulted  in  no  affective 
differences.    The  difference  between  the  two  studies  could  come  from  a 
variety  of  sources.    For  one  thing,  the  cartoons  used  in  the  present  study 
were  to  some  degree  "student  tested"  In  that  the  characters  were  familiar 
to  the  student  population  being  studied  since  they  appeared  in  a  regular 
cartoon  strip  in  the  student  newspaper.    The  nature  of  the  cartoons  used  1n 
the  Frei singer  study  were  not  specified. 

Of  great  Interest  was  the  significant  Interaction  between  Sex  and 
Presentation  Mode,  accounted  for  by  differences  in  comprehension  scores. 
To  hypothesize  that  the  males  were  distracted  by  the  cartoons  would  go 
against  a  fairly  stable  body  of  research  on  distractabll ity  (see  Halley, 
1975  for  a  review  of  some  of  this  litftrature).    One  possible  explanation 
worth  further  exploration  might  be  that  the  males  were  indeed  not  "distracted" 
by  the  cartoons  In  the  printed  text,  and  as  a  result  they  missed  visual 
imagery  cues  which  could  have  been  used  to  recall  specific  information. 
This  would  account  for  their  slightly  better  performance  in  the  audio-visual 
condition  since  the  slides  of  the  cartoon  were  not  spatially  related  to  the 
informative  presentation.    When  only  one  sensory  modality  was  used,  males 
scored  the  same  as  females  (printed  text)  or  higher  than  females  (audio  and 
visual  only). 

In  summary  and  conclusion,  the  results  of  the  present  study  would  seem 
to  suggest  that  if  comprehension  is  the  only  goal  of  instruction,  the  less 
expensive  printed  text  is  Just  as  effective  as  the  more  expensive  illustrated 
text  or  audio-visual  presentation.    If,  however,  there  is  a  concern  for 


the  student  to  enjoy  the  presentation  while  learning  the  information,  there 
Is  some  support  for  the  value  of  the  cartoon  niustrated  text.    The  overall 
equality  of  the  printed  text,  cartoon  niustrated  text,  and  aud1o-v1sual 
presentation  would  seem  to  make  final  choices  largely  a  matter  of  personal 
preference  and  budget  limitations* 


REFERENCES 


Adams,  W.  J.    Thi  use  of  sexual  humor  in  teaching  human  sexuality  at  the 
university  level.    Family  Coordinator^  1974,  23,  365-368. 

Baker,  E.  L.,  &  Popham,  W.  J.    Value  of  pictorial  embellishments  1n  a 
tape-slide  instructional  program.    AV  ConTnunl cation  Review,  1965, 
13,  397-404. 

Barr,  A.  J.,  Goodnight,  J.  H..  Sail,  J.  P..  &  Helwig,  J.  T.    A  User's 
Guide  to  SAS  76.    Raleigh,  N.C.:  SAS  Institute,  Inc.,  1976. 

Brooke,  L.  A.    Let's  keep  our  basic  business  students  interested.  BaUnce 
Sheet.  1973/74,  55,  173-174,  188. 

Dwyer,  F.  M.    Visual  learning:  an  analysis  by  sex  and  grade  level . 
California  Journal  of  Educational  Research,  1971,  22,  170-176. 

Dwyer,  F.  M.    Strategies  for  Improving  Visual  Learning.    State  College;  PA.- 
Learning  Services ,  1978.  ' 

Fre1s1nger,  S.  D.    The  effect  of  cartoon-imbelllshed  programed  textual 
Instruction  on  students'  skill  learning  and  affective  learning. 
Paper  presented  at  the  Association  for  Educational  Conmuni cations 
and  Technology,  1976.    ERIC  Document  ED  128  008. 

Gruner,  C.  R.    Wit  and  humor  in  mass  cormuni cation.    In  T.  Chapman  & 

H.  Foot  (Eds.),  Humor  and  Laughter.    New  York:  John  Wiley  &  Sons,  1976. 

Halley,  R.  D.    Distractalill  1ty  of  males  and  females  in  competing  aural 
message  situations:  a  research  note.    Human  Conmuni cation  Research, 
1975,  2,  79-82. 

Hance,  K.  G.,  Ralph,  D,  C,  &  Wlksell,  M.  J.    Principles  of  Speaking. 
Belmont,  CA.:  Wadsworth  Publishing  ComparVy,  1962. 

Hsia,  H.  J.    The  information  processing  capacity  of  modality  and  channel 
performance.    AV  Cbninuni cation  Review^  1971  ,  19,  51-75. 

Kauffman,  S.  P.,  &  Dwyer,  F.  M.    Effectiveness  of  cartoons  and  photographs 
in  in-service  training.    California  Journal  of  Educational  Research, 
1974,  24,  197-204. 

Larson,  C.  U.    Persuasion:  Reception  and  Responsibility  (2nd  ed.).  Belrnont, 
CA.:  Wadsworth  Publishing  Company,  1979. 

Lumsdaine,  A.  A.,  &  Gladstone.  A.    Overt  practice  and  audiovisual  embellishments 
In  M.  A.  May  &  A.  A,  Lumsdaine  (Eds.),  Learning  from  Films.    New  Haven: 
Yale  University  Press,  1958. 

Main,  R.  E.,  &  Griffiths,  B.    Evaluation  of  audio  and  pictorial  instructional 
supplements.    AV  Communication  Review,  1977,  Z5_,  167-179. 


530 


5 


Miner,  M.  E.    Charlie  Brown  goes  to  school.    English  Journal,  1969,  59,  1183 
1185. 

Neie,  V.  E.    The  lighter  side  of  science.    Science  Activities,  1973,  TO., 
35-37. 

Nie,  H.  H.,  Hull,  C.  H.,  Steinbrenner,  K.,  &  Bent,  D.  H.  Statistical 
Package  for  the  Social  Sciences  (2nd  ed.).    New  York;  McGraw-Hill 
Book  Conipany,  1975.  " 

Peter,  0.    A  sense  of  humor.    Folio,  March  1978,  pp.  41-54. 

Popham,  W.  J.    Pictorial  embelHshments  In  a  tape-slide  Instructional 
program.    AVCofmnunl cation  Review*  1969,  17,  28-35. 

Rein,  I.  J.    Rudy's  Red  Wagon:  Communication  Strategies  In  Con temporary 
Soc 1 ety.    Glenvlew,  IL.:  Scott,  Foresman  and  Company,  1972. 


Van  Mondfrans,  A.  P.,  S  Travers,  R.  M.  W.    Learning  of  redundant  material 
presented  through  two  sensory  modalities.    Perceptual  and  Motor  Skills, 
1964,  19,  743-751. 


APPENDIX  A 
Results  of  Factor  Analysis 


Factor  I 

Factor  H 

2 

h  ■ 

exc1t1ng/bor1ng 

.842* 

.156 

.73 

interesting/dun 

.853* 

,173 

.76 

humorous/sarlous 

.689* 

-.301 

.57 

6n joyabl e/ unen joyabl s 

MR* 

147 

75 

happy/ sad 

.697* 

.we. 

unusual/ordinary 

.768* 

-.112 

.60 

helpful/unhelpfui 

.045 

.907* 

,83 

1nformative/uninformat1 ve 

-.045 

.871* 

.76 

worthwhile/worthless 

.233 

.762* 

.64 

1nvol v1ng/uninvo1ving 

.591 

.304 

.44 

Proportional  eontributlon 
to  common  variances 

4.128 

2.447 

The  analysis  was  a  principal  components  with  varimax  rotation.  Each 
factor  was  required  to  have  at  least  two  items  loaded  at  .600  or  above  . 
with  no  secondary  loadings  at  .400  or  above. 


APPENDIX  B 
Cell  Means 


Female 


Male 


Row  Means 


Printed  Text 


Comprehension 

Enjoyment 

Helpfulness 


87.467 
7.267 
10.600 


88.267 
8.400 
10.134 


87.867 
7.833 
10.367 


Printed  Cartoon 
Text 


Cornprehens  1  on  91,467 
Enjoyinent  9.200 
He!  pf  Illness  10.867 


79.200 
9.467 
10.134 


85.333 
9.333 
10.500 


Audio-Visual 
Presentation 


Comprehension  79*467 
Enjoytnent  8.000 
Helpfulness  10.333 


80.533 
8.000 
10.667 


80.000 
8.000 
10.500 


Audio  Only 
Presentation 


Comprehension 

Enjoyment 

Helpfulness 


72.000 
5.333 
11.800 


79.200 
4.600 
10.733 


76.000 
4.967 
11.267 


Visual  Only 
Presentation 


Coniprehenslon 

Enjoyntent 

Helpfulness 


59.733 
6.667 
7.733 


66.667 
8.333 
9.000 


63.200 
7.500 
8.367 


Column 
Means 


Comprehension 

Enjoyment 

Helpfulness 


78.187 
7.293 
10.267 


78.773 
7.760 
10.133 
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THE  LIBRARY  AT  VIRGINIA  TECH 


The  Newman  Library  at  Virginia  Tech  is  there  to  serve 
the  needs  of  the  students  and  faculty  by  providing  the  tools 
for  the  basic  instruction  and  research  needs  of  the  universi 
community. 

To  help  you  rnora  effectively  and  efficiently  use  the 
resources  of  the  library^  we  would  like  to  introduce  you  to 
several  valuable  keys  which  can  help  unloclc  the  niysteries  of 
the  library i     There  are  four  major  Iceys  wb  will  talK  about. 

First  is  the  key  to  knowing  where  to  locate  the  books 
and  periodicals  you  will  need* 

Second  is  the  Key  known  as  the  card  catalog  system. 
Third  is  the  key  known  as  the  library  classification 
systems. 

Finally,  there  is  the  key  to  some  rules  and  services 
which  help  to  make  the  library  a  better  place  for  you  and 
everyone  else. 

Let's  begin  with  that  first  key-"lQcation  of  books  and 
periodicals . 

You  should  know  that  in  addition  to  the  main  Newman 
Library y  there  are  two  branch  libraries  with  books  and 
other  materials  in  specialized  subject  areas*     The  books 
and  resource  materials  in  branch  libraries  are  for  the 
use  of  all  students  * 

The  Architecture  Library  is  located  in  Cov^gill  Hall 
and  contains  materials  about  architecture  and  visual  arts, 
All  the  books  in  the  Architecture  Library  are  catalogued 
in  the  main  library,  but  after  the  usual  call  number,  you 
will  find  the  abbreviation  ARCH* 

You  will  find  the  Geology  Library  in  Derring  Hall.  Not 


everything  in  this  branch  library  is  written  on  rock^  All 
the  books  and  learning  resources  located  in  the  Geology 
Library  are  also  catalogued  in  the  main  card  catalog  of  the 
Nevman  Library,  but  after  the  call  number  you  will  find  the 
abbreviation  GEOL- 

Most  of  the  books,  of  course,  are  located  in  the  main 
Newman  Library,  and  we  will  focus  most  o±  our  attention  on 
finding  books  and  periodicals  there* 

In  the  main  library  there  are  three  subject  divisions: 
Himanities,  Social  Science,  and  Science  and  Technology, 

Humanities  books  and  periodicals  are  located  on  the  1st 
and  2nd  floors  of  the  Newman  Library.     Here  you  will  find 
books  and  periodicals    which  provide  you  with  information 
about  philosophy  and  religion,  history,  music,  language  and 
literature,  theatre,  and  communicatioris , 

Books  about  Van  Gogh,  Matisse,  Bach,  Beethoven,  McLuhan, 
and  Kierkegaard  will  be  found  in  the  Humanities* 

Did  you  know  that  the  library  receives  newspapers  from 
around  the  world  as  well  as  from  across  the  U,S.  and  the 
Commonwealth  of  Virginia?    These  newspapers  are  available 
in  the  Humanities  Division  of  the  2nd  floor. 

Books  and  periodicals  in  Science  and  Technology  ara 
located  on  the  3rd  floor  of  Newman  Library.     Here  you  will 
find  information  about  the  natural  sciences,  medicine^  agri- 
culture, technology,  mathematics,     and  military  science. 
This  is  the  place  to  come  if  you  need  a  solution  to  that 
impossible  equation,  or  have  you  ever  thought  of  building 
your  own  bridge? 

The  Social  Science  Division  is  located  on  the  4th  floor 
and  contains  books  and  periodicals  about  psychology,  law, 
geography,  anthropology,  political  science,  sociology,  business, 
and  education • 
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Lot's  begin  With  that  ftrau  Hey-Oie  location  of 
booKs  and  periodicaia. 

tm  ihBuid  know  that  in  addition  to  tb«  a«iti  S««b>4P 
Library    there  are  two  branch  Ubrarlea  With  and 
otoar  aatariala  m  specialized  suhject  wta..    The  bocks 
and  r,.oure«  Bat.riaig  in  Branch  Ubrariea  ara  feir  the 
use  of  all  students. 


Tha  Arehltoctur.  LlSrary  ia  loeat.d  in  Mwgill 
and  coBtaini  aawl.la  aboW  archlt.eturi  Md  visual  Mts- 
All  tBe  heoM  in  tha  Arehlt.cturs  Library  M.  flatalo^^d 
in  thfl  ■ain  librarv,  but  aitar  the  usual  call  nuabM,  -/ou 
will  find  thft  abbraviatiOB  AROI. 
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^  Librae  i-J^ 

in  tfiia  branch 
^11  the  Poofea 
d  in  th©  Geology 

,  but  #f ^eff  fc^a 


Keiit  of  th*  tooka,  o£  e^urse,  a^a  locate  in  ch^  ^in 
K^wman  Library,  and       will  £o«^  ^^^sn^^n  an 

finding  bQoka  and  periodieala  thsre* 

in  th^  «ain  library  tt^^e  ara  threa  aubj^^t  divMianai 
Hum^nitisa,  Social  Scienet,  and  iQienoe  and  Technology. 

Vto^sitiea  bOQHa^  and  periodicals  are 
lacated  un  tha  la*  ind  2nd  '.floora  o£  tne 
H^wmmn  t^ibrmry.    H^re  you  will  iif^d  Poafea 
and  periodicals  wttich  provide  yaU  wi«h 
informat^ion  about  philosophy  and  fmU^ion, 
history,  muaic,  languag^a  and  Uteracure, 
theacre,  an4  ce^unicationi .  ' 

mokm  a&out  Van  Gogh,  MatiJie, 
isethQvan,  Hciuhart,  and  KierKegaard  will 
ba  found  in  the  HuBanitlea. 
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Did  you  know  that  the  library  receives  newspapers 
froffl  around  %hm  world  aa  wall  as  from  across  the  U.S, 
and  the  Comfflenwtalth  of  Virginia?    These  newapapera 
are  ivailable  in  the  Humanities  Division  on  the  2nd 
floor. 


Bookj  and  periodicals  in  Science  and  TechnQlogy  are 
iQcated  on  the  3rd  floor  of  Se^naan  Lihrarv,    Hera  you 
will  find  infQrsatlan  about  Ehe  aaturai  sciancasj  msdicine,- 
agriciilture,  teahnQlogy,  matheaattqi,  and  mtlitary  science, 
lliia  is  thg  place  to  come  if  yoa  n^ed  a  aelution  ta  that 
iapoaaible  equation,  or  have  you  evir  thought  of  building 
your  own  bri^^? 


The  Social  Scienct  Division  is  located  on  the  4th  fleor 
and  contains  books  mnd  ptriodicals  about  piychology,  law, 
geography j  anthropology ^  political  aci#neei  aociology, 
businesa ,  and  gducation  # 

THEN  mSiRT  FILM  iKiTQ 
TW'UP  REEL, 


Several  other  typas  of  special  materiaia  are 
located  en  the  4th  floor  along  with  the  Social 
Science  Division- 

You  will  find  raost  of  the  roicrofoTOi?  (such 
as  microfilm,  microfiche,  and  miCTOcards)  located 
there • 


^Wa'LL  ST^^T  OUR 


There  is  alao  a  map  library^ 
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MEASUREMENT  AND  ASSESSMENT  TECHNIQUES 
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Media  specialists  and  Instructional  developers  generally  ask  three 
questions  when  the  measurement  of  attitudes  1s  promoted  as  a  component  of 
the  Instructional  design  process. 

s  Why  are  attitudes  important  In  the  teaching/learning  process? 

^  Why  is  It  Important  to  measure  attitudes? 

§  How  are  attitudes  measured? 
This  paper  will  attempt  to  answer  these  questions. 

Part  h    Importance  of  Attitudes  and  Attitude  Measurement 

When  the  Instructional  developer  designs  a  classrooni  activity  there 
should  be  at  least  two  categories  of  learning  outcomes  in  mind— those 
directed  toward  cognitive  goals,  and  those  related  to  the  attitudes  of  the 
learner.    There  is  little  need  to  discuss  the  rationale  for  the  Importance 
of  information  acquisition  by  a  learner  as  a  result  of  instruction.  The 
need  for  establishing  attitudinal  goals  and  for  planning  activities 
designed  to  produce  affective  outcomes  in  learners  as  a  consequence  of  an 
Instructional  sequence  is  a  little  more  difficult  to  explain  to  many. 
However,  it  has  become  increasingly  apparent  to  many  involved  in  educa- 
tional technology  research  that  one  of  the  major,  unique  consequences  of 
mediated  instruction  is  not  directed  toward  knowledge  gain.  Rather, 
instruction  from  television,  film  and  slides  often  produces  certain 
attitudinal  positions  in  students  not  necessarily  found  when  media  are 
not  used  in  teaching.    In  a  recent  review  of  educational  technology 
research  it  was  found  that  when  attitude  hypotheses  were  tested ,  over 
fifty  percent  of  the  time  desired  attitudinal  positions  or  changes  were 
produced  (Simonson,  1977;  1979ai  197gb),    In  other  words,  in  about 
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one  half  of  the  e)<peHments  reviewed,  the  attitude  toward  instructional 
method  or  content  area  possessed  by  students  were  either  favorable,  or 
changed  In  a  desired  direction,  after  instruction  that  was  delivered 
primarily  by  media. 

Still,  the  instructional  developer  may  say,  "So  what,  the  purpose  of 
instruction  Is  to  promote  the  acquisition  of  knowledge  and  skills,  and 
it  doesn't  make  any  difference  if  students  like  or  dislike  what  they 
learn,  or  how  they  are  taught." 

This  persuasive  and  prevalent,  if  narrow,  argument  is  somewhat 
difficult  to  refute.    The  most  powerful  rationale  for  the  need  to  promote 
attitude  positions  in  learners  would  be  to  demonstrate  a  direct  relation- 
ship between  attitudes  and  achievement,  or  Hking  and  learning.    And  as 
a  matter  of  fact,  numerous  researchers  have  identified  just  such  a  rela- 
tionship (Simonson  and  Bullard,  1978;  Simonson,  1977;  Levy,  1973; 
Fenneman,  1973-,  Perry  and  Kopperman,  1973;  and  Greenwald,  1966,  1965,  for 
example).    However,  most  educational  researchers  are  very  reluctant  to 
claim  that  there  is  any  cause  and  effect  linkage  between  these  two 
learner  variables. 

Probably  the  development  of  a  positive  attitude  in  learners  should  be 
a  desirable  end  in  itself.    There  are  several  reasons  why  attitudes  of 
learners  should  be  important.    First,  most  instructional  developers  would 
agree  that  there  are  cases  when  it  is  legitimate,  and  Important,  to  urge 
learners  to  accept  the  truth  of  certain  Ideas.    In  other  words,  to 
promote  an  attltudinal  position.    Second*  while  the  strength  of  the 
relationship  between  attitudes  and  achleviment  is  unclear,  it  makes  common 
sense  that  students  are  more  likely  to  remember  information,  seek  new 
ideas,  and  continue  studying,  when  they  react  favorably  to  an  Instructional 
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activity,  or  "'like"  a  certain  content  area,    Thlrdi  there  are  some 
Instances  when  influencing  the  attitudes  of  students  is  not  desirable,  so 
instructional  developers  should  be  aware  of  what  techniquos  do  affect 
attitudes.    In  this  way  possible  bias  can  be  recognized  and  eliminated. 
Last,  attitudes  toward  Instruction  felt  by  learners  can  lell  the  developer 
great  deal  about  the  inipact  of  that  Instruction  on  the  learning  process* 
In  other  words*  we  need  to  assess  the  opinions  of  our  students  toward  the 
learning  activities  we  are  subjecting  them  to,  1f  for  no  other  reason 
than  to  improve  the  quality  of  our  procedures.    Whatever  the  reason, 
attitudinal  outcomes  should  be  important  considerations  for  the  designer 
of  instruction* 

DEFINING  ATTITUDES  I    Before  attitudes  can  be  measured,  we  must  identify 

what  this  term  means,  and  what  contributes  to  addi- 
tional positions  in  learners.    Attitude  has  been  a  difficult  concept  to 
adequately  define,  primarily  because  it  has  been  defined  by  so  many,  but 
also  because  of  its  many  lay  uses  and  connotations*    One  of  the  earliest 
definitions  of  attitude  was  proposed  by  Thomas  and  Znaniecki  (1918), 
They  defined  attitude  as: 

A  mental  and  neural  state  of  readiness,  organlied  through  exper- 
ience, exerting  a  directive  or  dynamic  influence  upon  the 
indlvlduars  response  to  all  objects  and  situations  with  which 
it  is  related  (Thomas  and  Znaniecki,  1918), 

In  other  words,  while  attitudes  are  latent  and  not  directly  observable 
in  themselves,  they  do  act  to  organize,  or  to  provide  direction  to, 
actions  and  behaviors  that  are  observable*    Also,  attitudes  vary  in 
direction,  either  positive  or  negative;  in  degree*  the  amount  of  positive- 
ness  or  negati veness;  and  in  intensity,  the  amount  of  coniTiitment  with 
which  a  position  is  held  (Fleming  and  Levie,  1978), 
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Additionally,  attitudes  have  three  components:    affective,  cognitive, 
and  behavioral  (Zimbardo  and  Ebbeson,  1970).    The  affective  component  is 
said  to  consist  of  a  person^s  evaluation  of,  liking  of^  or  emotional 
response  to  some  object  or  person.    The  cognitive  component  is  concept 
tualized  as  a  person's  beliefs  about,  or  factual  knowledge  of,  the  object 
or  person.    The  behavioral  component  involves  the  person's  overt  behavior 
directed  toward  the  object  or  person. 

Part  II.    How  Attitudes  are  Measured 

Since  attitudes  are  defined  as  latent,  and  not  observable  in  them- 
selves, the  instructional  developer  must  identify  some  behavior  that  would 
seem  to  be  representative  of  the  attityde  in  question,  so  that  this 
behavior  might  be  measured-    This  characteristic  of  attitude  measuremerit 
1s  justifiably  the  most  criticized  limitation  of  this  area  of  educational 
evaluation.    However,  without  going  into  the  question  of  the  over-all 
validity  of  attitude  measurement,  there  are  several  generally  recognized 
procedures  used  to  determine  an  individuaTs  or  group's  attitude  toward 
some  object  or  person.    It  is  those  procedures  that  are  outlined  below. 

CHARACTERISTICS  OF  MEASUREMENT i    Before  procedures  for  measuring  attitudes 

are  discussed,  there  are  several 
characteristics  of  measurement ,  in  general,  that  should  be  considered  in 
order  to  determine  if  an  evaluation  technique  is  an  effective  one. 
Basically,  attitude  measures  should: 

•  be  valid  -»  In  other  words,  is  the  instrument  appropriate  for  what 

needs  to  be  measured? 
e  be  reliable      Does  the  measure  yield  consistent  results? 
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•  be  fairly  simple  to  administer,  explain,  and  understand  Generally, 

the  measures  that  yield  a  single  "score"  of  an  attitude  position 
epitomize  the  intent  of  this  characteristic,  although  the 
single  "score"  may  be  deficient  in  meeting  the  Intent  of  other 
characteristics  of  good  measurement, 

•  be  replicable      In  other  words,  someone  else  should  be  able  to  use 

a  measure  on  a  different  population,  or  in  a  different  situation, 
to  measure  the  same  attitude, 

CATEGORIES  OF  ATTITUDE  MEASUREMENT:    Basically,  there  are  four  categories  * 

or  approaches s  for  collecting  attitude 

information.    These  approaches  aret 

•  self- reports,  where  the  members  of  a  group  report  directly  about 

their  own  attitudes , 
i  reports  of  others,  where  others  report  about  the  attitudes  of  a 
person  or  groups 

i  soci ometri c  procedures ^  where  members  of  a  group  report  about  their 

attitudes  toward  one  another,  and 
n  records^   which  are  systematic  accounts  of  regular  occurrencas^  such 

as  attendance  reports,  sign-in  sheets ,  library  check-out  records, 

and  inventories. 

Within  each  of  these  categories  there  are  one,  or  more,  strategies  for 
measuring  attitude-related  behaviors,    Most  cofmonly,  attitude  reasurement 
is  accomplished  by  one  of  the  fotlowing  techniques, 

%  questionnaires 

•  rating  scales 
i  interviews 

•  written  reports 

•  gbservations 
%  soclometrics 


PROCESS  FOR  ATTITUDE  MEASUREMENT:    Any  attenipt  at  measurement,  including  the 

evaluation  of  attitude,  requires  that  a 
systematic  process  be  followed.    Such  procedures  will  not  guarantee  an 
effective  measurement,  but  they  do  increasti  the  likelihood  of  this  occurring, 
considerably.    Generally^  there  are  six  steps  to  be  followed  during  the 
attitude  measurement  process. 

1.  Identify  Construct  to  be  Measured  -  A  learner  could  conceivably  have  an 
attitude  position  toward  any  object,  situation^  or  person.    When  instruc- 
tion is  designed,  those  attitudes  that  are  important  to  the  learning 
activity  should  be  clearly  identified  and  defined.    For  example ^  1f  the 
developer  of  instruction  wanted  to  ascertain  the  impact  of  a  certain  type 
of  media  on  learners,  the  construct  "attitude  toward  instruction  by  film" 
could  have  been  an  attitude  outcome  that  was  evaluated. 

2.  Find  an  Existing   Measure  of  the  Construct  -  Once  a  certain  attitude 
has  been  identified,  the  designer  should  attempt  to  locate  an  instruinent 
that  will  measure  the  relevant  construct*    Generally,  such  tests  will  have 
been  tried  out  in  other  instructional  situations  and  should  include  some  state 
ment   of  reliability  and  validity.    Additionally,  instructions  for  adminis- 
tration of  the  test  selected  should  be  includes.    This  will  simplify  the 

job  of  using  the  Instrument  for  the  instructional  developer. 

The  most  obvious  disadvantage  to  using  a  pre-prepared  measure  is  that 
it  may  not  be  evaluating  the  specific  attitude  construct  being  studied. 
Even  if  this  is  the  case.  It  may  sometires  be  possible  to  extract 
valuable  information  from  an  instrument  designed  to  test  an  attitude 
position  similar  to  the  one  of  specific  interest. 

There  are  a  large  number  of  sources  for  finding  existing  attitude 
instruments.    Buros'  Mental  Measurements  Yearbook  is  probably  the  most 


wi^el/  kncMii.    Of  equal  value  is  Bufos'  Tests  in  Print,    Other  reference 
scujpc^s  fcr  attitude  tests  include  the  CSE  Test  Evaluation  Series  i 
Rohifi;  OH' g Measures  of  Psychological  Attitudes,  Rosen's  Attitudes  Toward 
Schn^  Ind School  Adjustment,  and  Knapp's  Omnibus  of  Measures  Related  to 
Sc^^  Jagid  Aktti tudes  (see  reference  list  for  citations), 
S.^^n^tg'uct  an  Attitude  Measure  -  Obviously^  1f  no  existing  measure  of 
th^  re^^  lev^fit  atti  tude  1s  available  the  instructional  developer  will  need 
to  ca  ^t^rijct  hi  s/her  own  test.    This  Is  probably  the  most  difficult  and  time 
coy^iijiif irig  step  1n  the  attitude  measurement  process.    Often  It  is  the  most 
critic       The  test  developer  should  apply  a  large  degree  of  patience  and 
effort  duHng  these  procedures, 

Cf  the  many  types  of  attitude  measurerent  possible  (see  above one 
te^SiiiHye  widely  used  that  seems  to  possess  most  of  the  characteristics 
of  a  g-^W  Msure  is  the  "Agreement ,  or  Likert-type,  Scale."  This 
tsthrH^e  lises  statements  about  the  attitude  object  that  are  either 
cl^arlJ  f^^^^^^T^  or  unfavorable.    Each  subject  resporids  to  each  test  item 
accord  ^"^q  to  his/her  perceived  attitude  "Intensity"  toward  the  staterent, 
Ofteii^  stU^lents  are  asked  to  answer  each  test  Item  on  a  five  point  scale 
th^t  h^^  responses  that  vary  1n        degree  of  agreement  to  the  statement 
f^m  s  ^^origiy  disagree  to  strongly  agree  (see  Figure  1).    One  advantage 
of  till  ^  te^linl que  is  the  ease  of  scoring  and  surrmarizlng  the  information 
obtaHn^^.    (For  a  complete  listing  of  the  steps  necessary  to  construct  an 
ag^e^ni«efit  scale  see  Table  1;  for  a  more  complete  description  of  how  various 
tyKes      stUtude  measures  are  designed  see  Henerson ,  1978.) 

yom  a  test  Is  constructed  locally    it  is  critical  that  reliability 
an^  va  '^dfity  Information  be  collected  for  the  measure.    Of  these  two  con* 
cBpts,  VaTlltlity  (1e.  appropriateness  of  Instrument)  Is  the  most  difficult 


to  deterniine*    Validity  for  a  test  depends  on  a  number  of  factors,  such  as 
the  type  of  test  and  its  intended  use.    Basically,  there  are  four  categori 
of  validity, 

#  Construct  validity  (or  the  extent  to  which  you  can  be  sure  a  measure 
represents  the  attitude  construct  whose  name  appears  In  its  title) 
can  be  determined  by: 

1.  opinions  of  judges, 

2.  correlations  to  other  measures  of  the  same  constructs 

3.  measures  of  criterion  group  subjects  (those  who  obviously 
possess  the  construct) i  or 

4.  appeals  to  logic, 

•  Content  validity  (or  the  representativeness  of  the  sample  of 
questions  included  in  the  Instrument)  is  usually  detenTiined  by 
careful  analysis  of  the  items  in  the  test.    There  Is  no  simple 
process  to  determine  content  validity  other  than  a  close,  thoughtful 
examination  of  each  itein  separately*  and  all  items  together* 

i  Concurrent  validity  (or  the  agreement  of  a  test  with  a  parallel  form 
of  the  test  on  the  same  topic  that  was  administered  at  approximately 
the  same  time)  is  determined  by  correlating  the  results  of  two 
parallel  measures  of  the  attitude.    This  correlation  coefficient  is 
reported  as  an  index  of  concurrent  validity, 

§  Predictive  validity  (or  how  well  a  measure  will  predict  %om  future 
behavior)  is  determined  by  comparing  results  of  an  attitude  test 
to  some  reasure  of  behavior  given  at  some  point  In  the  future, 
Againt  this  type  of  validity  is  usually  expressed  by  a  correlation 
coefficient  found  by  coniparing  results  of  the  two  measures* 


Obviously,  determining  validity  is  not  a  simple  task.    However,  every 
instructional  developer  who  constructs  a  test  of  any  type  should  be  acutely 
aware  of  the  need  to  develop  valid  instruments.    Since  there  is  no  single 
established  method  for  determining  validity,  the  test  originator  should  use 
care  1n  constructing,  administering  and  interpriting  tests,  and  their 
results.    Finally j  these  precautions  should  be  described  to,  and  for,  the 
consumer  of  the  test  data. 

Reliability,  or  the  ability  of  a  measure  to  produca  consistent  results, 
1s  usually  less  difficult  to  determine  than  validity.    Thfire  are  several 
methods  of  determining  reliability  that  can  be  easily  used  by  the  attitude 
test  developer.    The  "Test-Retest"  method  Involves  the  re-administration 
of  the  instrument  to  the  target  group  and  correlating  the  results.  The 
"Split-Half"  method  uses  an  arbritrary  division  of  the  instrurnent  Into 
two  halves.    Results  from  each  half  are  correlated  and  reported  as  a 
reliability  coefficient.    "Alternate-Form"  reliability  Involves  the 
correlation  of  the  results  of  two  parallel  forms  of  tests  of  the  same 
attitude  construct.    Each  subject  takes  each  fDnm  and  the  resulting 
correlation  Is  reported  as  a  reliability  estirnate. 

Each  of  these  techniques  will  yield  a  score  from  0.00  to  1.00.  The 
higher  the  number,  the  more  reliable  the  test.    Generariy  speaking  * 
reliability  coefficients  above  .70  are  considered  respectable.  Scores 
above  .90  are  not  unconnon  for  standardized  attitude  tests.    As  with 
validity,  the  results  of  reliability  estimation  should  be  reported  to  the 
consumer  of  your  testing  activities.    (For  more  information  on  validity 
and  reliability  estimation  see  Anastasi,  1968;  Cronbach,  1970^  Fitz-Gibbon, 
1978;  Henerson,  1978;  or  Talmage,  1976). 

4.    Conduct  a  Pilot  Study  -  While  It  is  possible  to  obtain  validity  and 


reliability  data  during  the  actual  testing  portion  of  the  instructional 
activity,  it  is  much  more  logical  to  at  least  try  out  attitude  instruments 
before  they  are  forrriany  used.    This  should  be  done  In  order  to  obtain 
appropriate  data,  but  also  to  uncover  minors  but  troublesome,  administra- 
tive problerns*  such  as  mi sspel lings,  poor  wording,  or  confusing  directions. 

5.  Revise  Tests  for  Actuil  Use  -  Results  of  pilot  testing  should  be  used 
to  revise,  or  "polish,"  attitude  instrufitents.    Once  the  ''bugs"  have  been 
eliminated  the  measure  Is  ready  to  be  used  with  its  Intended  target  audience 

6.  Summarize,  Analyze^  and_  Display  Results  -  When  testing  is  completed 
resulting  data  must  be  interpreted*    Generally,  attitude  test  results 
should  be  handled  similarly  to  any  other  test  information.  Obviouslyi 
numerical  data  is  easier  to  manipulate  than  verbal  Information.  Whatever 
attitude  responses  have  been  collected,  it  is  Important  to  sunmarize, 
analyze  and  display  the  results  in  such  a  manner  that  they  are  easily 
and  quickly  understood  by  others.    For  example,  raw  data  should  be 
collected  and  recorded  on  s^i  type  of  surmary  shcjet.    A  "quick-tally" 
sheet  is  often  used  when  data  Is  to  be  hand  scored  (see  Figure  2).  When 
data  is  to  be  machine  scored  It  should  be  recorded  on  the  familiar  80 
column  coding  sheet.    This  sheet  corresponds  to  the  standard  80  column 
computer  card.    An  even  simpler  technique  for  coding  data  is  to  have 
students  respond  to  attitude  test  items  on  a  "mark-sense*'  (optical 
scoring)  sheet.    When  this  type  of  scoring  sheet  is  used,  the  raw  data 
can  be  easily  accessed  for  coniputerlzed  statistical  analysis  without  the 
need  for  Intermediate  coding  steps  (see  Henerson,  1978), 

After  data  has  been  collected  and  coded  it  should  be  analyzed.  Most 
researchers  consider  "Agreement  Scale"  data  to  be  "Ordlhal -Scale" 
(Ferguson,  1971),  so  it  can  by  analyzed  using  standard  tests  of  description 


and  Inference.    However*  data  about  instructional  mode  or  content  area  1S 
often  useful  1f  it  is  merfly  averaged  and  compared  to  other  averages*  In 
other  words >  did  the  class  change  in  average  "Attitude  Toward  India^'  after 
viewing  the  film?   Did  they  react  favorably  to  "Film  as  a  Method  of 
Instruction  as  Compared  to  Ucture?'^    Often  merely  '-eyebaning"  average 
scores  or  "change  scores''  will  be  helpful  in  analyEing  data  and  in  inter^ 
preting  the  impact  of  fp^-H-httd  Instruction  on  attitudes. 

Displaying  data  1s  another  effective  method  of  analyzing  it  for  the 
consumer*    Charts  *  graphs,  &  bar  diagrams  are  examples  of  data  display 
techniquas  that  are  useful  in  assisting  >n  the  understanding  of  what  test 
results  indicate.    Whatever  thtb  process,  tha  developer  of  an  attitude  test 
should  make  every  effort  to   decipher   the  results  of  tha  measure ,  and 
to  explain  those  conclusions  and  impl ligations  to  the  consumer  of  the  data* 
Sometimes  this  may  mean  that  the  developer  will  have  to  apply  a  considar- 
able  amount  of  subjective  interpretation  to  data.    This  Is  not  necessarily 
bad*    Naturally,  the  more  that  opinions  can  be  based  on,  or  replaced  by^ 
facts,  the  more  powerful  conclusions  will  be,    (See  Figure  3  for  examples 
of  data  display  techniques.) 

Conclusion 

Obviously,  attitude  reasurement  is  only  one  of  many  evaluation  technique 
for  the  Instructional  developer  to  consider  for  use.    However »  since 
attitudes  are  "predispositions  to  respond*'  they  would  seem  to  be  related*  in 
some  manner  and  degree,  to  what  happens  to,  and  in,  a  learner  during  the 
instructional  process.    Admittedly,  there  are  many  possible  deficiencies  in 
the  measurement  techniques  described.    Attitude  measurement  Is  cartainly 
not  as  refined  as  anyone  would  like*    That  does  not  mean  that  understanding 


the  impact  this  construct  has  on  the  learning  process  1s  not  important.  In 
order  to  facilitate  a  more  ccmiplete  undirstanding  of  tiaching  and  learning 
the  Instructional  developer  should  collect  as  much  relevant  data  about 
that  process  as  is  possible.    Attitudes  and  attitude  measurenient  are  a 
critical  component  of  that  understanding. 
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FIGURE  1  -  AGREEMENT  SCALE 


1  2  3  4  5 

SA  A  U  D  SD 

Strongly  Agree  Uncertain  Disagree  Strongly 

Agree  Ctsagree 


ERIC 


FIGURE  2  -  "QUICK-TALLY"  SHEET 
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Tlu*  uiilcrcd  sculc  v'MJstsis  nt  n  riilkciuifi  uf  !vUilt*nic;Us  ih,il  t'S^c^  j 
IJIIi^C  Ul"  opliiiuils  uhnllC       Ullihiilw  uhjcci; 

•  Sdinnt  t^*ui.-|ics  \oii  tilings  iliui  help  in  guitiiiL*  ;i  job. 

•  Mnbi  iuuclicr^  dwoply  cuiv  iihoiii  thuii  MiiJcnts. 

•  SnMicuinc^  bwlu>ul  can  Iw  iiiiciehiini!. 

•  SwIiMul  IS  rcully  a  lu^ic. 

•  There  arc  tcsu  nuifn  riile>  ut  scliuuK 

•  Schuc)!     the  inobt  vuhiahle  way  I  wliii  bpoiul  in\  !iiiic. 

Hcinio  inwliuiun  uii  tlic  niciisuic,  these  suigtnenis  ureus5iL.hey  j  s...s,. 
v;iUie  alofig  un  uitiUide  coniiiuiiiiii  fn  u  p.Micl  ufjiulycs.  An  nhtMi;;\-?n'  ib 
tliefi  cuiistruetej  svisiwii  presents  the  slulcnients  hi  i;nidt)in  urdei  i:i.?/u: 
nidie.ilniy  their  scuk  ^ulties.  The  respnnd^iit^  ale  usked  tu  the^jk  inti:^ 
st;itwMnenlN  with  whuit  thes  u^ree.  Only  iiphiinn  $iii:ciiwiU\  urc  ;uw>i. 
l^uumd  sfufcnicNfs  iir.*  imm  inu'liidcd  ni  tlnH  kind  *if  nu';nine,  sinct'  T\'iple 
with  ihif-fini:  utiitndes  wnuki  agree  lu  u  stjicinefit  of  hicl.  Note  &.A  t!  - 
icspondenib  urc  usked  I*' select  only  ilhi-^c  iiei  they  u\:r  .  uii  t 

in  rejetit  ull  ulhi^rs. 

Steeps  For  CoiiNifiicUng  nnd  Unhig  nn  Ordered  Scale 

I.  Accnnnilute  u  !ar;ie  nuinhi»r  nf  stuieineiits  ahoiil  (he  aUilude 

(nppruxiinuicly  10())  A  ^'Uul  MJnue  inr  llu^u-  siuleinenN  \m:  '  : 
f'ffcfVUHvs  of  pftKpt'cUve  {esiJOikls^iils.  Make  siuc  lli:il  ih.-  Ma!  ;^  vn' > 
lepicseni  ii  svide  r:inL:e  nl  opiiiiinh  mehuliiii^  niiulcralc  ones. 

1  PLice  ungh  stiilenieni  on  n  scpaiale  punv  ()i  papei. 

-  Selccl  a  grijnp  nl  I'tjiies  <pje|M;iU!v  or  rnou^  penple  h>;^  'he 
pfn:ipe.live  fespondeMi'^i  .nid  u^k  e.Kli  -uic  Insna  llu  ^Uiternen';  nif^* 
M  piles  ruiiUMiif  U^^\\  h!*ihly  nninvuiat>lc  (M  ihiough  iwnii.i!  i'H  m 
hitzhU  javuruhle  ^  1 1  Miikc  wrc  thr  /uJi:vs  titHhrs/uHJ  that  jrc 
vluHsiJym^  the  statcmvnn  uml  hhi  inJnutUig  rhulr  u^rcvtncnt  r  dtv- 
ii\irL\  nH  tti  with  tliL  Ul 

4.  Throw  out  sUlcni^nis  that  havu  been  plactfd  in  widely  din^^iing  piles. 
These  are  uinbi^iunib  statenicnts, 

5.  Fur  each  reinuininy  ^tulcnicnl.  arrive  at  a  scale  value  by  cuntpuliiig  the 
median  or  nieun  ptmiiiuii  uf  thu  assignntwnts  by  the  judges. 

u  Select  a  set  uf  statemenls  (under  25)  whose  seiile  values  will  give  you  a 
spiend  that  evenly  wovcrs  the  eonlinumn  fruin  hiylily  unravorable  tu 
hiizltlv  favorable, 

7.  <\ii»strncl  the  t|iieHliuniunie  by  lisiini:  the  stalenients  m  randimi  onii  t\ 
Do  nut  iiulicale  scale  ^jlues  on  the  nistiunien!. 

Adininisler  the  nisliitnieiU.  instruetuig  rejipondcnis  tu  indicate  with 
winei?  stutenienis  llwy  iiiiiee. 

Cnniptne  a  seme  loi  each  ies|u)ndent  liy  nnJing  the  mean  of  tltc  scale 
values  ol  the  statements  selected  l>y  that  respondent. 


kfs  fur  C'fiiisiriicUng  ;inU  Usiny  ii  StNiiniiMc  Diffuri^iiilnl 

I  1)^  ii/  Muinc  iIh'     iiiulw  >  on  wrJi  to  ijivosijpniL'. 

!  Sdc^  l  ;;pf^inpiijfc  iiJjcciisv  pnin  (uppi  iSiinJit-l)'  \  ni  Yimi  iiiiiy  uish  (u 

?i'k'Wi   tri^ii)  iIk  list  pfiniiiccl  M  \lw  ciiJ  ul"  lliib  cluiplci  ut  litiin 
list  if  ii  suiu  ynur  suidciiis=  Sow  niay,  on  Hie  iJiIicr  lumd, 

wisii  Id  nuikc  lip  ynur  uwn  lisi. 

JHiiU'  the  iiltitiuk'nhtcci  oi  phuH  .li  iht*  inp  uC  (lie  |vjgc  uiuf 

I'Li^jt-  chc  lidjuctivcs  beiwulii      h  yiUi  urc  exLiiiiining  iiitjrc  (luin 
.liiiUidc  ulijcwi.  use  ihi^  %t\\\\M  JMi'*c'"e  order  fiir  cucli  niiitiido  f^bjcci, 
ktfvp  llie  wdrdb  ill  the  sai^   v  i><iiion,  jVuvide  "randiijii  pohnity  " 
This  incnns  (hat  tliu  udicelivc  pai/s  iliuuld  //m/  he  liHicd  so  ilini  all 
positive  rchpuines  l.Jl  un  one  hide  un*^  all  iiegruive  fespuiiswH  nii  ihe 

'UluT. 


Jh   X  on  i*n4'  ^^f   Uii'  ^tvi-r   Mni-s  hi'CW4j.cMi  im{  ft  priir. 


!  I   i  i  i  i  M  I  Ml!  r  f  i  1-ImI  1  i 

t  ■\  \  f  !  iili  ii  I  r 
ir* 

fru'nUiv              _    _        _       _  unfriiMiilly 

fair   ^         _    ^  unfnir 

5Qur                _      _  _   _  ^weec 


liiMruwt  ihu  rcspiMidcius  ahuiit  liow  ;iiul  u'lierc  iu  mark  ilicir  liiiinps. 
TlicN  slUHiiJ  lie  iiistrucu'd  \o  respond  qiuekly  und  uii  iiie  hwm  ol  ilieif 
Itr^i  i iiipreFsions.  ^'oii  \\\uy  find  iliui  soiiie  pci.)p!c  jr?f  not  etiinruriiihle 
K'^plOid^1L:  In  ii  eniieep!  (e.g.,  plu  sk:il  cdueaiioiK  art)  with  seeiiilngly 
iii.ippii>pinjfe  ntljeelives  If.^.,  sweet-suur).  You  will  liuvciu  rcussure 
siieli  peMple  iluii  this  typi?  of  bcnlc  culls  for  nnprvssinn^^  imH  studied 
fc^pitiibes.  If  you  are  UMiig  tile  iiisininieni  vvltii  ehildr^n,  vqu  would  do 
\\el!  in  Liive  Iheni  soiiiC  priietiee  uiiji  (iihcr  enuei^pls  (iMIiuicJc  i)!>jeetH) 
Iviou^  ihes' iiinke  their  uitlnps  on  iheeoficepts  ViHi  \vuni  to  iiiciisuru, 
^  Ciiiiipiite  II  peisoirs  seoie  by  iJssignin^  .I'M"  in  ieHpunsis  iiuliijiitiii^  ilie 
\m\    Jiei*.iiive  leNpofise.  n  ''7"  in  ilie  tnnsi  posiiue  lespuiise,  niid 
sviniMu  liiii'MfUHliiite  fespiMises  ronn  "2"  !>»  "o"  ;iee.)f  dni|;ly .  A  persiHrs 
\cuie    loi   unv  line  atlUiide  ('ItjCet  is  ilie  .iveKiue       lespniiNCS  lo  tliu 
jilifiulo  p,ll  IS. 

Hill'  ,iiw  nuMe  adieciive  p;ins  yuu  ini^lit  n^e  Im  ennni lueiuii:  seiiuniiic 
gllni-n  I  ud  r  :il  nii;  ^e;ileb: 
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NEEDS  ASSESSMENT 
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Assistant  Directur 
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Needs  assessment  is  a  fundamental  part  of  evaluation^  and  yet 
it  has  not  developed  as  rapidly  in  sophistication  as  other  evaluation 
mfethodologles ,     This  is  largely  due  to  the  difflculcy  in  measuring  real 
needs s  as  opposed  to  wants,  interests ,  etc*      This  paper  discusses 
some  of  the  crfrical  issues  in  needs  assessment y  and  outlines  some  of 
the  fundamental  quest ionii  that  mu^r.  be  answered  in  order  to  design 
a  valid  and  reliable  needs  assessruent  study*      Emphasis  is  placed  on 
the  need  for  considering  resources  and  constraints  in  designing  a 
realistic  approach  to  needs  assessment. 


NEEDS  ASSES S>CENT 


Needs  assessment  is  a  large  class  of  activities  that  are  alTred 
at  determining  the  needs  that  exist  among  a  certain  group        people,  Some^ 
cirAes  these  needs  are  obvious  and  readily  observable;  othertimess  these 
needs  are  hiddGn  and  not  perceivable  without  fine  instrumentation*     If  we 
assume,  as  I  think  we  must,  that  all  prograi.^s  eKist  to  serve  people  with 
needs  5  then  the  importance  of  needs  assessment  can  be  seen*      In  essence, 
needs  assessment  is  the  '^front-end"  of  evaluation ^  and ^  as  Hichael 
Scriven  (1978)  has  said,  "it  puts  the  value  in  evaluation/*  Evaluation 
f   upletes  the  needs  assessment  by  determining  trie  extent  to  which  needs 
have  been  met  by  a  program  intended  to  meet  those  needs . 


ment .       It  is  much  easier     o  assess  the  effectiveness  of  a  single  program 
':nan  to  probe  the  ill-delined  realm  of  human  needs*     Evaluation  focuses 
ca  a  single  object*  while  needs  assessment  focuses  on  assessing  a  vir-- 
tually  infinite  domain  of  needs.     One  of  the  factors  that  lias  led  to 
Che  relatively  slow  development  of  the  technology  of  needs  assessment 
has  been  the  fact  that  "needs"  (real  needs)  are  so  difficult  to  measure. 
Like  attitudes,  motives ,  and  personality  traits »  they  are  generally 
hidden  below  the  surface  of  everyday  awareness,     It  is  rarely  possible  to 
receive  a  reliable  answer  to  the  question:   "What  are  your  needs?'* 


We  know  much  more  about  evaluation  than  we  do  about  needs  assess^ 
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The  most  frequently  used  methods  of  needs  assessment  use 
questionnaires  which  ask  that  very  question.     The  problen  is  that  few 
people  are  aware  of  their  real  needs,  or  else  r^^^al  needs  are  confounded 
by  perceptual  biases.      Often  symptoms  of  needs  are  confused  with  real 
needs.       For  instance  *  the  person  who  says  ha  needs  more  money  might 
only  want  more  moncyj  or  money  might  be  a  substitute  for  a  lack  of 
self-esteem  (or  some  other  internal  need) *      Attempting  to  discriminatG 
between  wancsj  interests^  needs,  and  other  related  characteristics 
has  been  the  mnst  significant  thorn  in  the  side  of  needs  assessors 
I'^^r      .1  wAg  time*      When  educators  ask-  "I^That  courses  do  you  need?''  or 
'Mh<r.  skills  do  you  need  to  learn?",  it  Is  virtually  impossible  to 
distinguish  these  felu  .leeds  from  interests  and  wai  ts,     As  a  result,  and 
to  the  extent  that  time  and  resources  perTnltj  conscientious  needs 
assessors  are  using  unobtrusive  measures  of  nes^dj  such  as  archival  records, 
observation  of  people  at  work^  performance  t*--^cs,  rather  than  u'  Ing 
questionnaires  exclusively.    The  belief  is  that  infemd  needs  are  more 
legitimate  th.r.n  self --reported  aeeds.      However,  as  long  as  the  limita-- 
tions  of  questionnaire,  self-report  methods  are  understood,  there  is 
no  harm  in  using  them  as  part  of  a  needs  asfiessment  effort. 

The  rest  of  this  paper  will  deal  with  some  of  the  other  critical 
issues  In  needs  assessment  which  are  integral  to  developing  an  valid  and 
reliable  ne^ds  assessment  instruFientation,     Bear  in  mind  that  thern  is 
no  one  correct  methoJ  of  needs  a  cessment,  and  the  correct  mathod  for 
each  assessor  must  derive  froni  actual  informatioi'  requirements* 


Wiat  tjrpe  of  jijieds  are  yc  ■  i  intarested  ln_  assess^inj^? 

Are  there  dlf iclencias  in  performance,   :  kills i  attitudes ^  motiva^ 
Cion?      It  is  vital  that  the  needs  assessor  havfe  some  hypotheses  con^ 
earning  the  apparent  nature  of  the  need.     These  hypotheses  will  determine 

•  niost  appropriate  n.etho^  'S  of  needs  assessinent ,     If  there  is  a  per- 
formunca  deficiency ,  it  Is  important  that  performance  be  evaluated.  If 
thcira  is  an  ''interest-need" ,  then  a  questionnaire  might  be  in  order,  Hy-- 
pothesizing  about  the  presumed    nature  of  the  need  makes  it  possible  to 
detarmine  a  strategy  for  assessing  the  need*     Without  such  hypotheses ^ 
needs  assessment  would  ba  like  shocr^ing  in  the  dark* 

Wiat  is  the  cunteKt  of  the  need  to  be  assessed? 

Although  we  all  try  to  be  idealistic  in  needs  assessment ^  and 
as  objective  as  possible^  we  must  recognize  the  realities  of  the  situa-- 
tion.      An  educational  organization  is  looking  for  needs  that  can  be 
addressed  by  educational  means ^  and  the  needs  assessor  would  not  stay 
employed  lone;  if  he  was  not  able  to  identify  educational  needs  for 
"     r:^^anizat  ion .      We  should  also  be  aware  of  the  political  - pressures 
la:    nay  impact  on  the  needs  assessment  process,     ^.^lo  is  powerfrd  in 

organiEst ion?    \n\o  should  be  involved  In  the  needs  assessment  process? 
From  the  earliest  stages  of  needs  assessment >  the  problems  of  imple-- 
nientation  must  he  anticipated  and  addressed.       Tlie  context  of  ne^ids 
assesBment  shouLd  be  careful^  -  defined  if  the  needs  assessmen^=  results 
an',  to  have  any  impact  at  all* 
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t#iat  resource  g  are  available  for  need  susses  sment  and  subsequent _^_g_tlon? 

A  fruitful  needs  assessment  must  be  based  on  a  realistic  judg^ 
ment  of  resources.     We  must  know  early  in  the  proGess  what  type  of  (and 
how  intensive)  an  effort  the  organ  if  ^^nt  ion  is  willing  and  able  to  under^ 
write.      In  addition,  to  what  extent  is  the  organization  willing  and 
able  to  satisfy  naeds  that  might  be  assessed?       It  is  the  most  fundamental 
rule  of  needs  assessment  that  there  must  be  commitment  to  meet  with 
appropriate  action  any  needs  that  are  uncovered.     Otherwise ,  needs  as-- 
S'Cssment  will  become  nn  eKerclse  in  frustration.    As  Scriven  (1978) 
has  explained,  needs  assessment  and  action  planning  to  meet  anticipated 
needs  should  be  part  and  parcel  of  the  same  process , 


How  does  one  go  about  jgyelopinp      plan  for  needs  assessmen t ? 

In  developing  any  needs  assessment  plan  it  is  essential  that 
the  following  questions  be  addresst  -  r 

1.  For  whom  is  the  needs  assessment  i,\ tended?  Information 
should  be  collected  In  an  appropriatt   form  for  this 
person  or  persons, 

2.  Ifho  is  part  of  the  target  pupulation?     Although  this  might 
seem  ^'Mickey  Mouse--s  a  frequent  failirg  in  needs  assess- 
ment Is  an  inadequate  awareness  of  the  client  population, 

3.  What  data  collection  method s  shou]^  be  used?    The  answer 
to  this  question  depends  on  tha  hypothesised  nature  of 
the  need  and  resources  available, 

4.  Sho!^      sampling  tachniques  be  used?     It  is  rarely  necessary 
to  uwjarve  all  members  of  the  target  population.  Often 

a  small  random  sample  of  persons  will  suffice  and  decrease 
cost  considerab 1y , 

5.  Who  should  be  involved  and  how  should  they  be  involved? 
It  is  rarely      possible  to  do  a  comprehensive'^  needs  asset^' 
ment  alone  ^  and  the  help  of  othfc'r''  in  invariably  tequired 


6.  What  are  the  anticipated  costs  and  who  needs  to  approve 
the  plan?      Requirad  approvals  of  the  plan  and  costs  should 
be  accomplished  as  soon  as  possible  to  facilltata  revisions ^ 
if  necessary^  and  to  secure  coimnitmenL  for  the  project, 

7.  What  constraints  might  hinder  the  needs  assessment?     Just  as 
it  is  Important  to  anticipate  resources ,  it  is  also  imp.^rtant 
Co  anticipate  constraints.     This  way  we  can  be  prepared  for 
the  inevitable  contingencies  which  might  hinder  the  proj  ect » 

How  ghouid  the  Information  be  disseminated  and  used? 


The  needs  assessor's  job  does    not  end  with  data  collection. 
It  is  part  of  his  duties  to  present  the  Information  to  appropriate  de^ 
cisiQn--raakers  and  make  sure  that  the  information  is  being  used  appropriately. 
Those  who  are  not  used  to  making  decision&-  based  on  needs  assessment 
daca  might  have  great  difficulty  in  doing  so  without  help  and  guidance. 
The  closeness  of  the  needs  assessor  to  the  data  makes  it  impt^ratlve  th.it^ 
he  participate  in  the  development  of  action  plans  to  meet  assessed 
In  addition  J  it  is  essential  that  the  needs  assessor  supervise  a  deb^ 
ing  to  determine  the  ef f ectivene.^is  of  the  process  as  used  and  to  suggest 
.f^ros^emen ts  for  the  future. 


Metds  assessmenr  is  a  complex  process,  the  full  complexity  of 
which  is  Just  beginning  to  bp  recc^iized.       It  is  not  Just  a  matter  of 
objective  data  collection  and  analysis.      This  paper  has  endeavored 
to  explain  soma  of  the  complexities  and  suggest  methods  for  dealing 
with  them.      It  will  be  Bome  time  before  the  technology  of  needs  assess- 
ment  is  as  sophiBt    :ated  as  evaluation  technologies.      However,   it  is 
essential  that  this  be  the  case,  since  evaluation  is  so  integrally  tied 
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to  needs  asaessment*      You  a^e,  it  is  ridl .= '^tltjua  to  put.:  nuyti  -i:^:?danr.v  jn 
the  evaluatioii  of  a  ptogram  that  might  hava  illegl*  ImnL  :  justification 
for  its  eKistence.  y  programs,  if  adequately  asseoJed  in  terms  of 

needs  satisractlon,  might  never  have  seen  the  light  ci  day. 
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A  visual  literaQy  test  was  constructed  following  ths 
Standards  for  EduoatlQnal  and  psychQlQglcal  Tests  ^  visual 
literaGy  was  deflnid  and  the  crltloal  attributes  to  be 
teBted  listed  In  a  Table  of  SpeQlf loations ,     visual  literacy 
vfH*B  shown  wlthiii  tiie  cultural  envlronmeat  whiah  reflects 
man's  orientation  to  a  world  of  video,  and  placed  in  the 
verbal-visual  modal  repreaeated  by  left  braln-right  brain 
hemisptLtric  research,     A  pilot  test  was  run.    Analysis  cf 
this  data  was  used  to  oonstruct  the  final  form  of  the  test 
which  was  given  to  a  stratified  random  sample  of  Eduoatlonal 
Supervisors,  librarians  and  Media  Tectaologists  in  the  State 
of  Missouri*    Results  were  reported  and  a  i^est  Manual  and 
Manual  for  AdministratiD^  QOnstruQted,p 
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visual  LiteraQy  Asscissmeat 


!•  INTRODUCTION 

Th3  mode  of  eommunlGation  used  in  teaching  j^nriuences 
iLarnin,^  and  retintioa.     In  the  1950's        J.  Phillips  of 
Thn  University  of  Texas  suggested  some  differences  In  re- 
tention related  to  modalities  of  learaiag*    Reading  was 
the  least  efficient  while  saylns  and  doing  or  hands-on 
experience  was  the  most  infl^^mtial.     Kinder  (1973)  quotes 
Phillips  telling  us  that  ^^Reteation  of  what  Id  learned  is 
likewise  related  to  sense  experience^     ohservation  and 
research  tend  to  show,  holding  time  as  nearly  constant  as 
posslhle,  that  people  generally  remembers 

10  percent  of  what  they  read  ■ 

20  parcent  of  what  they  hear 

30  percent  of  what  they  see 

go  percent  of  what  they  hear  and  secj 

70  percent  of  what  they  say 

90  percent  cf  what  they  sa;^  as  they  do  a  thing" 
This  may  be  interpreted  on  a  CLntlnuum  froE  passive  to  active 
involvement  in  learning  or  froL  the  symbolic  to  the  actual  or 
real  world  experience.     It  further  suggast^  that   the  closer 
the  facslaile  to  the  real  world,  thfi  greater  tha.  retention 
cf  learning. 
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Ted  0*  Gobim  suggeatidl  in  1963  that  "we  Isara  about 
]  per  cen^t  through  taste  and  another  1g  per  cent  througii 
the  sense  of  touch*     The  sFnse  of  smell  provides  about 
3h  per  cent,  and  hewing    provides  about  11   per  gent  of 
what  we  learn.     An  overwhelming  83  per  cent  of  our  learning 
is  through  visual  experiences," 

Marshall  McLuhan  suggested  in  1967  that  "any  mder-- 
standing  of  eocial  and  cultural  change  is  Impo^'elble  without 
a  knowledge  of  the  way  media  work  as  environments*"  Tele- 
vision has  the  capacity  to  bring  us  all  into  oue  cultiire, 
one  world  with  common  views  and  conoepts.  could  become 

the  great  levaler  nesatlng  real  differences     atween  us. 

-rh.^^  Trend  Report  put  out  quarterly  in  vf^ishington  D.G. 
foraua,^ts  social  change^     In  January  of  this  year  It  re- 
ported wfj^r  Bocinl  changes  as  ^ur  country  becomes  a  society 
whose  mu^x-  D^coduci'  lo  Information,    Already  fift^  percent 
of  our  work  force  Is  engaged  in  processing  Information*  By 
contrast  that  figure  was  only  10  percent  in  1950,     It  further 
suggested  tr:/'^  If  you  are  looking  for  a  new  career  you  shovilG 
try  information  dissemination  * 

BUu  this  is  not  a  new  prediction.     In  a  book  written  in 
1973  and  titled  informatiQn  in  1935  a  i^ey  forBcast  suggested 
that  "by  1935  there  will  be  4  to  7  times  as  niuch  scientific 
and  tectoical  Information  as  in  1970,"    Further  the  demand 
will  ou..y  be  satisfied  by  an  increasingly  accelerated  auto- 


nation  which  will  increase  100-fold  over  the  presgnt  ajstems 
Finally,  it  vrai  sugge,ated  that  the  progreas  of  automation 
itself  may  be  inadequate  to  keep  up  with  new  inf ormatioay. 

SO  the  Qiodes  of  information  dissemination  are  changing: 
reading  and  resaar chins  hooks  is  giving  way  on  the  ona  h^d 
to  computer  storage  a^d  retrieval  of  information  and  on  the 
other  ha^d  to  video  systems,    IJQt  only  .U'        h^.vQ  Qommerolal 
television  broadcasts  and  publi        '^evlslon  systems  but  the 
ability  to  recQrd  off  the  air  programB  to  be  retrieved  at 
will*    AS  yet  these  are  in  several  formats  as  in  the  early 
stages  of  audio  casaatta  recording  but  we  are  even  now 
thinking  in  terms  of  5D  imaging  which  will  add  yet  anothar 
dla:en3lon  of  reality* 

*riils  proliferation  of  information  and  modalities  of 
dl;ii3 ealnation  are  forcing  us  to  take  another  look  at  learn*^ 
Ing^  QOmmuni cation^  psychology  and  Hociologys     Theories  of 
mass  Qommunioatlon  assume  greater  Importaiice  than  ever  before 
The  hue  and  cry  of  parsnts  and  PTA^s  concerned  about  the 
Influence  of  sex  and  violence  on  telavision  and  the  effect 
of  advertising  on  young  minds  with  regard  to  Jur 
is  witness  to  this  fayt. 

It  is  important  to  ask  cursel^es  ..hy  this  Is  so^  j^u 
we  do  so  and  search  the  literature  for  an  answer ,  we  find 
a  growing  interest  in  the  effects  of  visuals  i     Moreov^er^  a 
concept  of  visual  literacy  has  been  developed  during  this 


decade,  and  research,  from  the  physiological  arena  suggests 
a  tie  to  differing  sensitlvltes  of  brain  hemlspheras: 
ths  left  side  of  the  brain  Is  sensitive  to  temporal  coaGepts 
of  order,  logic,  time,  writing,  verbalization  and  reason; 
the  right  side  of  the  brain  is  sensitive  to  i...Bges,  space, 
visualization,  invention  and  Intuition, 

During  the  decades  that  followed  the  invention  of  the 
printing  press  the  left  hemisphere  attained  dominance  as 
learning  was  almost  entirely  print  oriented  and  the  de- 
finition of  an  educatfcd  person  related  primarily  to  the 
ability  to  read  and  write  using  alphabets  most  of  which 
utilized  symbols  that  In  no  way  reflect  what  they  reprasent. 
Visuals  on  the  other  hand  are  pictures  o;'  the  real  world 
albeit  in  two  dimensions  until  the  disoovary  of  the  holo- 
graphic process  using  the  movement  of  light  through  time 
and  spaoe  to  create  3D  imagss* 

We  are  living  in  an  exciting,  changing  world  which 
demands  changing  conoepts  and  nndars tanding  of  the  processes 
which  affect  us*     One  of  these  processes  is  the  means  of 
acquiring  information.     School  is  by  no  means  the  only 
place  children  acquire  information  or  learning  although 
it  is  tha  primary  source  of  formal  learning*  Television 
is  by  far  the  greatest  source  of  informal  learning  and 
prsschool  children  are  the  single  largest  teievlsion 
audience  In  America  according  to  Marie  Winn  (1977) • 


"They  spe^d  a  greater  aumber  ojf  total  hours  a^d  a  greater 
proportion  of  their  ^akiag  day  watching  telev^ialotL  than 
any  Qther  group  p'^  she  says,     in  1970,  the  average  was  30,4 
hours  each  yfemk  and  the  figure  was  going  up.    By  thi  time 
a  child  graduates  from  iigh  school  he  or  she  has  spent 
many  more  hours  In  front  of  a  television  tube  tlis4  in  school. 
Why?    what  Is  the  appeal?    is  It  Importaat?    should  teachers 
become  profiqlent  at  utilising  this  medium  of  ooma^lcation? 
Should  there  ba  or  will  there  be  a  move  to^fiirfi  aore  and  more 
dlssemlrLatiQn  of  information  visually  as  ¥©  moire  from  the 
age  of  the  printing  press  to  the  reoord  and  cassette  press? 
What  Bkilla  will  be  neecLed,  If  indeed  this  is  the  case? 

We  have  moved  into  the  realm  of  noa^'Verbal  oorami^i cation 
Many  boolca  have  been  written  about  body  languagep  the  effects 
of  dress  upon  suocess^  photoanalysls »  Bubllaiiial  imfluenoes, 
embads^  synesthesia  and  techniques  devlsad  or  utilized  to 
brainwash  us*    Thm  media  spends  huge  aaoimti  of  moaey  to 
determine  the  precise  effect  of  television  eoMunlcation 
upon  specif lo  groups*    Motion  pictures  are  now  produced 
following  research  of  aGceptabillty  and  appeal  to  the 
pubLlQ,     rnflueaclng  people  thjough  the  visual  aode  is 
becoming  a  prealse  sclenGe  based  upon  practical  research, 
^iflth  a  greater  Involvement  of  the  senses ^  greater  Influence 
can  be  everted  with  longer  lasting  effects  upon  retention 
as  suggested  earlier. 


Jeffrey  Schraak  (1975)  suggeata  that  "we  kavfe  put 
vast  control  over  our  children's  minds  lata  tha  faiLada  of 
broadcaaters,  toy  and  food  maau.f actures ,  aai  Qther  cooiaercial 
Intereats  whose  dominant  ooacern  is  what's  good  for  profits 
and  sales,"    Nor  la  this  true  of  cMldrea  alons. 

Wllsoa  Brym  Key  (1972,  1976)  suggests  tfciat  we  as 
adults  are  teiag  sexually  manipulated  through  subllainal 
seduction.    His  books  attempt  to  tell  us  tha  aeeret  ways 
advertlslag  mea  arouse  our  desires  to  sell  theiK'  jroducts. 
Nor  is  %hXa  effect  only  from  ads  per  se.    Materia,!  Is  now 
being  incorporated  into  the  satartainmeat  or  program  section 
of  televlaioa  to  gain  the  sami  effeetB,, 

This  is  not  a  new  art.    Games  of  percsptloa  sad  hidden  " 
Images  known  as  anamorphlo  art  have  existed  froa  the  re- 
aalssaace  to  the  present  (Leeaan,  1975),     But  taday  the 
technology  exists  to  utilize  the  effects  as  nevex  before. 

Fortunately,  we  can  coabat  these  effects  by  an,  aware- 
ness of  the  processes  used  to  laflueace  us.         nay  also 
utilise  theae  processes  to  enhaace  our  comnuflioatLoii  siills 
with  others,    These  are  some  of  tha  reasons  for  'beconing 
visually  literate.     But  what  is  involved?    What  are  soma 
of  the  basics  of  visual  literacy? 

Doadla  (1973)  has  taken  an  artistic  approach  and  re- 
cognizos  three  levels  of  aay  visual  nessage.-    repr eaintatloaal , 
abstract  and  synbollc.    The  abstract  level  refers  to  the 


redUQtlon  of  tha  visual  to  its  baalo  visual  elements;  the 
symbolic  suggests  the  same  senas  that  priat  is  aymboUo: 
that  is.  ve  have  attached  meanins  to  speclflG  symbols. 

Others  have  takea  this  same  tack  and  attempted  to 
define  visual  oommunication  by  Its  elemsiits.    And  these 
are  important  as  any  artist  fcnowa.    However,  a  perusal 
of  the  llteratura  suggests  other  dlttenslons  such  as  sub- 
llmlaals.  embeds,  perspective,  lllualon,  layout,  cues,  and 
Bpeclal  attributes  of  some  formats  such  as  film  and  video 
tape  which  allow  movement  and  time  manipulation,  goae 
eleaenta  have  special  payohological  effects,     color.  Una 
and  camera  angles  might  be  Included  here,    finally,  the 
ability  to  operate  the  equipment  and  to  utilize  it  maximal- 
ly Is  important.    As  one  becomes  more  adept  at  equipment 
synchronization  and  utillzatloa  various  coablnations  of 
eiulpment  offer  added  dloeasloaB  of  comfflualcation  visually. 
A  good  example  of  effective  aultl-medla  utilization  occurred 
durlag  the  Blosntenaial  year  when  the  Corjs  of  laglneers 
carried  their  display  to  every  part  of  the  nation.  Most 
if  not  all  Of  you  saw  it.    yinally,  awareaesa  of  techalques 
such  as  polarization  allow  one  to  extend  the  capablUtlys 
Of  softwares,     slides  or  transparencies,  for  example,  oan 
sho'.  movement.    A  classic  example  is  a  traasparenoy  showins 
the  beating  of  the  heai-t  and  flow  of  blooi  through  the  body. 
The  heart  beat  oan  evea  be  regulated  on  tHe  spot  to  match 


that  of  aay  individual  watGhing. 

Slaae  all  of  ttiese  Gouoepts  s^ffeot  visual  coaffluni cation 
in  a  major  way,  a  Table  of  Specifications  vras  co^atruoted 
to  tacludi  them  for  the  piixpQss  of  test  coas truGtion. 


II.       mrOSS  AI^D  RATICNMiE 


The  purpose  of  this  study  (Turaer,  1978 J  was  to  con- 
struct a  visual  llteraoy  test  and  staadardlze  it.  Visual 
literacy  was  defined  and  ths  critical  attritutes  to  be 
teated  ware  chosen  after  carifully  reviewing  the  literature. 
Tliese  categories  were  listed  In  the  Table  of  Specif icatioas 
which  was  used  as  a  guide  for  the  teat  ooastriictloa,  visual 
literacy  was  shown  within  the  cultural  eaviroament  which 
reflects  man's  orieatatlon  to  a  world  of  video,  and  placed 
in  the  verbal-visual  model  represented  by  left  brala-rlght 
brain  hemispherio  research,    Utarature  was  qiioted  to 
support  the  concept  that  visual  information  is  more  com- 
plicated  and  broader  in  Its  deflnltioa  thaa  verbal  informa- 
tion.    It  is  associative.    We  must  become  faolllar  with  its 
elemeats  before  moving  into  a  oomposltlonal  stage.  also 
need  to  recognize  the  psyoho-phjrsical  forces  operatlag  In 
human  peroeptloa  since  visual  meaalng  laheres  in  all  senses 
and  not  just  la  seeing,    whole  chunks  of  inf ormatioa  can  be 
assimilated  through  sight  and  peroeptloa  with  great  speed. 
Moreover,  the  control  of  the  final  effect  of  a  visual  lies 
in  the  manipulation  of  Us  eleneats  and  the  planned  use  of 
visual  techniques. 

The  ooacepts  of  visual  literacy  chosen  for  this  test 
were  grouped  into  a  5  x  4  matrix,    '^ithla  the  5  categories 
3  sub-categories  were  embedded,     nsual  litaracy  for  the 


purpose  of  this  test  was  deriaed  as  the  ability  of  peoplg 

of  all  ages  to  read  visuals  as  they  read  printed  materials. 
It  includes  the  ability  to  write  visually  as  well  using 
graphicg,  oaaeraa  and  other  devices.    l"t  reaogaizes  that 
visual  literacy,  unlike  print,  is  multi-dimensional  and 
perceived  through  the  senses.    Words  are  sometlmea  ouBb&r- 
some  but  a  visual  oaxL  cQmaunicate  at  a  glance,  arouaing 
in  us  parts  of  our  own  nature,  feelings  and  s Qnaibllitles , 

The  teat  itself  was  deslgaed  to  provida  a  comprehanslv 
assessffient  of  the  ability  to  communioate  viaually,    it  is 
appropriate  for  students  In  high  school  and  oolLese,  for 
teachers,  librarians,  media  specialists,  and  supervisors, 
Bnphasis  is  placed  upon  one's  ability  to  read  and  construct 
meaningful  visuals. 

It  is  constructed  to  yield  a  dependable  meaaure  of 
Spearman's  "g."    The  single  score  obtained  sum^rizes  per- 
formanoe  over  a  wide  variety  of  test  materials.     There  is, 
however,  an  emphasis  upon  visual  skills  rather  than  verbal. 

We  make  no  distinction  between  what  Is  being  measured 
and  spearman's  "g."    The  difference  is  the  means  of  measurin 
it-  that  is,  the  type  of  item.    Intelligence  or  infornatlon 
of  any  kind  is  the  same  by  whatever  means  of  communication 
unless  the  means  itself  is  Gonsldered  to  be  intelligence. 
Like  Otis-Lennon  then  we  accept  the  fact  that  we  are  measur- 
ing one  general  thing  which  may  be  broken  into  many  factors. 


Oura  is  a  factor  little  recognised  until  the  age  of 
eleotronlG  media  even  as  priat  was  not  a  factor  u^tll  the 
development  of  a  printing  press  that  could  mass  produce. 
Media  concentrates  information,  syntlaesiges  and  GOMunioates 
it  in  a  total  seas 6,  enviroMient* 

Ideation  I.  notation  and  oomaunicatloa  come  together 
In  visualization:  >ie  neid  therefore  a  test  to  assess  more 
than  verbally  orieated  Intelligance  measuree,     The  domain 
uoed  is  based  upon  dimenelons  mentioned  la  visual  literacy 
and  media  research. 

The  test  may  be  used  as  a  basic  sompBteney  test  for 
the  basic  languaglaag  skills  of  visuals* 

Another  valid  use  would  be  to  diagnQse  Ixtdlvldual  visual 
oommuni cat ions  problems  of  teachers,  school  librarians  and 
Jiiedia  speQlalistSp    Unfortunately,  even  ^edla  speQlallsts 
today  have  looked  too  long  upon  media  as  hardware  lastead 
of  a  communicatlQn  medium  with  all  the  po^ar,  nuanaes,  and 
oadence  of  any  commimication  medium.    Therefore,  they  have 
failed  to  capitalized  upon  the  available  potential  of  visuals. 
This  test  should  demonstrate  laclc  of  languaglng  Bkllls 
within  the  fifteen  dimensions  encofflpassftd  by  the  test* 
These  may  be  grouped  into  five  factors: 
1 p  non-verbal 

2.  attending  devices 

3,  theoretical  applications 
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4.  meaaing  of  elemeats 

5.  ffledia  conpOBeats 

.inothar  use  of  tie  visual  literacy  tist  might  be  to 
□  ertlfy  that  faculty  oenbera  pQasess  the  basic  sicills  of 
visual  eoaimunlcatioa. 

Still  another,   to  show  the  public  the  responalbla  and 
acoouatable  .a^er  of  taacMng  th,  whale  student  through 
both  verbal  and  visual  communication  chamela  •  we  are 
attempting  to  remedy  the  liBitatioa  of 
basic  skills  via  one  midiua. 


measuring  or  teaching 


VISUAL 


"g" 


lon-varbal 
Attending  devices 
Theoretical  applicationa 
Meaning  of  elements 
Media  coaponents 


Virbal  reasoning  ' 
Verbal  conpreheasloa 
Flgural  reasoning 
Quantitative  reasoning 


Figure  1 .     The  divlsioa  of  s 


spearmaa's  "g"  into  visual 


and 


Verbal  componentE 


Ill*       DSVBIOPING  TEST  SPSQIFICATI0H3 


Researchers  agree  on  %hm  Importance  of  the  later- 
aetion  of  tliree  mala  classss  of  tasonomlc  variables- 
media  attributes,  Individual  differences,  and  outcoma 
behavlQrs*     But  media  attributes  are  often  defined  in 
terms  of  equipment  or  operatlone,     Ivor  K.  Davles  (1973) 
shows  that  major  trends  are  discernible  in  the  research 
literature  that  relates  cognitive ,  affec  ive  and  psycho- 
motor  classes  of  learning  objectives  to  these  categories^ 
However,  Richard        Clark  (1975)  suggests  four  posBlble 
schemes  to  describe  and  seleot  relevant  media  attributes 
that  Interact  with  indl visual  differences i  one  reasgas 
from  extant  trait  systems  like  Guilford's  structure  of 
the  intellect  I  another  from  cognitive  and/or  affective 
process  description;  a  third  from  typlGal  or  stypiQal 
learning  outcomes  a  la  Bloom -s  taxonomy i  and  the  last 
suggests  media  researchers  should  extract  media  attribute 
from  treatments  employed  in  previous  media  research*  The 
latest  reaearoh  shows  the  interrelation  of  languaging  and 
literacies  related  to  the  split-brain  syndrome  >rlth  the 
use  of  phonemes  and  morphemes  on  the  one  hand  and  kineses 
and  visemQs  on  the  other.     Larger  elements  of  visual 
vocabulary  include  color,  shape,  line,  light,  texture^ 
pattern,  perspective,  etc.    There  are  also  ways  to  e^presc 


oneself  visually,  types  of  visual  expression,  lllusioas 
and  visual  games  which  involve  learning  to  see  and  perceiv 
and  are  InfluenGed  by  time,  culture,  experlenGe  and  person' 
allty.     ImageB  that  go  beyond  surface  levels  Involve  the 
sensory  reglBters  without  mioh  we  would  have  no  motion 
in  films  or  TV.     Therefore,  the  content  of  this  test 
exajnines  the  cognitive/af f  eotlve  aspects  of  non-verbal 
oommunication  and  InfluenQee 


Identic  i.ppll- 

floatlon^  Theory    cation  .  Teotmolog.y 


Non-verbal • 

Body  language  2  2 

Subliminals  2  i  ^ 

illusions  2  2 

Devioes i 

layout  2  1  i 

pormat  cues  2  1  i 

Attention  g  1  i 

pllm/ l^apes  i 

Teach  5  ^ 

Movement  2  2 

Time  manipulation  1  3 

Eleraents ; 

Golor  2  2 

Line  2  2 

Camera  angle  12  1 

Media : 

Types  2  2 

CombinationQ  2  2 

components  4 


Table  1,     Table  of  Specifications. 


Item  types:    The  test  uses  non«verbal*  attendina:. 
theoretical,  meanlflg  and  media  componeat  type  questloas. 
Thiaa  include  body  language,  subliniaals ,  and  illusions. 
A  second  catigory  oontaias  layout,  format  cues  and  atteatloa 
gaiaing  techniques,    a  third  suggests  awareaess  of  under- 
lying  theoretical  applicatioas j  the  way  film  teaches,  how 
to  create  movemeat  and  oaaipulate  time.    Ressarober  in- 
vestlgated  effects  of  color,  line  and  camera  angle  are 
included  as  are  aspects  of  media  coaoepts  which  relate  to 
individual  dlffereacea. 

Levela  of  _  dif f  Iciaty :    Item  difficulty  was  held  to  aa 
average  of  ,5  or  50^  to  oaximlze  reliability.    Any  items 
with,  discrimination  indices  greater  than  .8  or  less  than 
.2  vers  deleted,    A  spiral  format  withia  each  quadraat  was 
used  J  that  is,  there  is  a  cycle  through  the  types  of  items 
four  tines  in  the  test, 

yiifflber  of  Items;     There  are  four  q.uastioas  in  each 
sub«eategory  of  five  major  classif icatloas  for  a  total  of 
sixty  questions.    This  holds  the  test  length  to  an  hour 
and  Is  adequate  to  assess  visual  literacy  concepts  since 
each  will  be  presented  four  times.    It  Is  a  5  x  3  x  4 
design , 

Oraaai  zatioa ;     jhe  test  is  segmented  into  four  equal 
quadrants.     This  meani  that  half  of  the  teit  could  be  given 
la  thirty  minutes,  or  a  mini-test  could  be  given  in  15  or 


20  minutes  by  using  one  sectioa  of  the  larger  test*  Thim 
allows  Several  ways  to  anal        the  data. 

Test  Items :     Visual  acy  experts  may  be  asked 

at  a  later  date  to  write  u^.  b  Iteas,    You  are  in  fact 
iiavited  to  do  so*     It  Is  planned  to  develop  a  large  test 
pool  of  items  to  be  used  in  future  test  construction  and 
development  * 

Method :     The  developmeat  of  the  test  Inaluded  the 
creatlori  of  the  test  speGificatlons,  writing  the  test  items 
compiling  the  prellmlnai'y  test  form,  trying  out  the  pre- 
liminary test  form  on  a  sample,  pretesting  and  analyzing 
the  item  statistics,  compiling  the  final  form,  admlnlsterln 
the  final  test  form  for  standardisation  and  preparing  the 
test  manual  to  be  used  with  the  test*    All  of  the  steps  in 
the  QonBtruction  of  the  test  meet  the  standards  for  Eduoa* 
tional  and  pByohQlogioal  Tests,        class  of  media  produc- 
tion students  was  glvett  the  prellDiinary  form  of  the  test 
for  item  analysis.    Using  these  statistics,  the  final  form 
of  the  test  vras  prepared.     A  raadozi  sample  from  the  popula- 
tion of  iduoatlonal  Supervisors,  librarians.  Learning 
ResourGe  and  Media  TeGhnologlsts  la  the  elementary  and 
secondary  schools  of  the  State  of  Missouri  was  used  to 
standardige  the  test*     Data  analysis  includes  mea^  SGore, 
median,  standard  deviation,  s eml'- Interquartile  range, 
Kuder  Richardson  20,  standard  error  of  measursaent  and 


Internal  consls-tmcjr .    a  Wteit  w&s  jeTformea  to  deteraiae 
-whether  ox  act  thele  was  a  slm^iflcaPat  diffgr-eaai  tit^isn 
the  two  graupB.  depeadLent  va^latLe  >fas  the  test  sooi^e 


IV.       RESULTS  AJD  DISOUSSIOJT 


T3ie  pilot  tfiit  ladlcated        average  Itea  aiffloultjr 
of  .38  •    By  drcpjtng  oiit  items  with  dlJfiauLty  less  than  ,2 
tl^s  dirfflcuLtr  l£vel  was  rats  si  to  *53  i^hich  met  the  pre* 

4werags  aascra^aLaatloa  vras  .2  which,  is  aQceptalDle  for 
cl^ssrpoia  testa, 

Reliability  of  the  test  was  ^65^  which  jLaoed  It  in 
th^  typiaa^i  jrang©  oj  classroom  tests,     ?lthln  categQries 
araaiabLLtty  SGores  iffrra  aiuoh  higher  ccnfiralBS  that  we 
had  ih^ludad  severai  Gpncepts*    Non*?erbal  and  devices 
categories  hsid  toe  loifsst  correlation  LaternBLly  but  still' 
cDS'rela.tlQsa  lat^malljr  exceeded  ,60,     jha  other  categories 
ot  film.ytapa  eLiaiints  a-nd  media  oorrsl&tad  ia  the  ninstlas 
The  results  pon^fljaed  our  espeetatiOB  that  oaly  bits  and 
pleoea  ^ra  ^Q^g^  abou^  visual  literacy.     Kflowledga  reflected 
toy  coi^reotly  aaweired  luestions  related  specif loalljr  to 
ireciulred  course  asterials.     It  is  our  bili#f  tbat  the 
results  oi  this  ssae  test  without  bilng  changed  would 
differ  signlf Icaa^ly  if  given  to  visual  llteraay  experts, 
Jt  le  hypothesised  that  there  would  h&  lao  questions  with 
"no  cflr^ect  ■  mm^mrs^  mid,  in  fact,  thess  aatd  quistlons 
shoi?i3ag  the  loweat  correlation  on  this  pi^rtlcular  analysis 
wore  perfectly  query  the  conoept  of  visual  literacy  per  se. 


Forn  B,  a  30  iteffl  teat,  was  compiled  fiQai  Lnformatiott 
acquired  from  the  pilot  test  and  was  givett  to  a  stratified 
raadoa  saople  of  the  populatioa  of  iducatloaal  supervisors, 
Librarlaas,  learning  Resource  and  Media  Technologists  In 
the  state  of  Mlasourl  with  a  questionnaire  to  be  used  for 
generallzatioa  of  test  results.    The  same  types  of  analysis 
Were  performed  ca    tne  data  and  oomparlsoaa  betveea  groups 
tested  were  made* 

Sixty-.two  peroent  of  the  saaple  reaponded.     Ihls  is  an 
unusually  higii  percentage  for  mall  ciuestlonaalres.     of  the 
91  vho  replied,  16  submltttd  inoomplete  data,    ihare  were 
75  usable  tests-  24  were  from  Educational  Supervisors  and 
51  from  library/Media  Jrof Bssloaals ,    lach  district  of  the 
State  of  Missouri  wag  represented  by  respond  eats  from 
counties  within  the  districts. 

The  male/female  dlatrlbution  showed  a  greater  pereeat 
of  males  in  the  Supervisor  group  and  a  greater  prepQnderaaoe 
Of  females  la  the  Llbrarf /Media  group.    A  t-tast  showed 
significant  male/female  difference  at  the  ,1  level  but  not 
at  the  .05  level. 

There  was  a  good  dispersion  of  scores.  Beliablllty 
for  the  30  Itea  fast  was  ,47  but  using  the  SpiBraaa^Brown 
prophecy  formula  to  determine  reliability  for  a  test  of 
45  similar  Items  would  raise  reliability  to  .59,'  60  similar 
Items  wouia  raise  It  to  ,64-  75  itens  to  ,69;  and  a  test 


of  90  simllsr  test  items  wnuld  Inorease  reilabiLitr  to  ,73. 

Still  aAotlier  way  to  liiorease  reliability  would  be  to 
eliminate  or  revise  the  questions  vitii  correlations  less 
than  .3.    Theia  were  three  of  these  la  our  sample.  Of 
course,  a  better  approach  may  be  for  subjects  to  be  cone 
familiar  with  these  concepts.     ThXa,  too,  would  change  the 
item  oorrelatioa.     Commeats  suggest  uafamiHartty  with 
subject  matter  may  be  the  great ist  ooaslderatloa , 

The  standard  error  of  measuremeat  la  an  eatliaata  of 
the  probable  exteat  of  error  in  the  test  scores  and  is 
interpret  id  in  the  same  manner  as  s  standard  devli,tloa.  i. 
standard  error  of  measurement  of  2.5,  for  example,  suggP.sts 
that  for  any  particular  teat  score  the  odds  are  a  to  1 
that  the  subject's  true  score  (the  average  score  on  several 
similar  tests)  would  not  deviate  from  the  obtained  score 
by  more  than  3,5  points = 

The  more  reliable  and  error  free  a  test  Is,  the  smaller 
the  error  of  aeasiirementi  that  Is,  the  standard  deviation 
of  the  error  dlstributloa.    The  staadard  error  or  Eeasureaent 
for  this  test  is  2.1,  a  relatively  small  error  of  Eiaaureaent. 
This  suggasts  that  the  resBarcher  can  place  confideace  in 
the  accuracy  of  the  test  scores,     jt  means  that  2  out  of  3 
times  a  subject's  score  In  visual  literacy  will  fall  within 
the  range  of  the  obtained  score  on  this  test  +  g.l  p'^ints. 
Factors  iafluenclng  reliability  arei 
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1  ,     the  length  of  the  .^eat — the  loager  the  test  of 
similar  Items,  the  higher  the  reliabil.ltjr  estimate. 

2.  less  variability  la  the  populatioai  that  Is,  a 
smaller  range  of  iadlvldual  differeaQes  or  talent  decreases 
reliability. 

3.  unf ami liar ity  with  conteat  also  reduces  reliability. 

4,  variability  of  contant — tha  aore  similar  the  Items 
are  lad  coatent,  the  greater  the  Intircorrelatlons,  and  the 
higher  tha  reliability  eatlnate.    Items  vlth  dissimilas' 
oonttnt  have  lower  or  zero  oorrelatlona . 

5,  Iteo  difficulty — selection  of  iteas  of  modtrate 
difficulty  fflaxlmizes  total  test  dispersion.    The  higher  the 
Item  dispersion j  the  higher  an  Item  correlates  with  other 
iteius.    However,  it  is  more  important  to  s elect  similar 
items  than  It  is  to  coatrol  item  difficulty  to  maxlialzQ 
reliability. 

Item  analysis  of  dlscrimlaatlon  and  Alffloulty  were 
again  rim.  For  a  detailed  explaaatioa  sea  Turner,  1978, 
pages  111  through  114, 
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V.  VALIDITY 


NO  test  review  io  conslete  without  Information  atout 
its  validity:  that  Is.  the  degree  to  ^hich  a  test  measurss 
Wbat  it  Is  suppose  to  measure,     cat tell  (1964)  suggested 
that  test  validation  appears  la  a  QQatinuum  from  praotlcal 
to  conceptual  valldltyi  tHat         ty^nn^  ^^t^^^*^^ 
oa  tiae  test  itself.    Purposs  defines  the  method  of  valldatlaa 
to  %e  used,     "with  coatent  mlidity, "  says  Liake  (1976). 
"the  ooncepta  to  be  measured  a?e  spsolfled  bjr  the  test 
developer."    This  «ag  doae  la  the  rablB  of  Specif ioations 
provided  by  the  researcher.     LeMke  coatinues  that  content 
validity  Is  provided  by  a  logical  analysis  of  the  doaaln 
of  Items  as  was  doae  y,Xth  tiie  pilot  test,  with  the  final 
foro  of  the  test  belag  ooastructed  aocortiag  to  Item  analysis 
statistics  from  the  bH 

construct  validity,  on  the  other  haaa,  is  cocGemed 
vrith  traits  eabidded  in  the  test  and  the  degree  to  which 
these  account  for  performance,    Oorrelation  analysis  caa 
be  wsed  to  establish  the  goastruct  vali Uty  of  a  test. 
Unfort'oaately,  no  other  visual  llteraoy  test  exists  to  date 
against  which  this  test  can  be  correlated.    To  this  research- 
ei-'s  taowledge,  no  staadards  have  been  established  for 
csdiated  Instructloaal  expertige  against  which  this  test 
aigbt  be  compared.    The  fielfi  of  visual  literacy  itself  is 


new  haviag  developed  wittila  this  decade. 

Tlie  other  type  of  valldatloa  wita  which  the  ressarclier 
might  be  Interested  Is  criterion  validity.    This  type  of 
validity  can  be  eatabliahad  by  axpectaaoy  tahLas  and  simple 
correlations  with  appropriate  variables  whan  thesa  exist. 
Sinoe  visual  literacy  Is  a  new  field,  content  validity 
a  jemed  the  more  aDDrooriatp.     mhis  i«  ■, ^„ 

light  of  the  stand  by  the  Rational  iducatloBal  Assoalatioa 
with  regard  to  all  teats  being  a  type  of  achlevemsat  test. 
It  is  recognized  by  test  coastruotioa  experts  that  oQntent 
validity  Is  most  appropriate  for  achievement  tests.  More- 
over, crlterloa  validity  taices  time  to  develop  slaoe  It  la 
based  upon  predictive  statistics. 

Reliability  la  a  necessary,  but  lasuffloleat,  condltioa 
for  criterion  validity.    Moreover,  the  naxlmun  value  of  the 
validity  caefflcleat  Is  the  square  root  of  the  reliability 
coeffioient.     The  correotlon  for  attenuation  estimates  the 
correlation  between  true  scores  of  the  test  and  the  criterion 
and  allows  an  estlmatloa  of  validity  If  the  ffleasureaeat 
lastrumeat  is  perfectly  reliable. 


VI .       HEC0>MEfTD4i:  I  Oil  3 

it  was  shown  that  a  visual  literacy  test  can  be  con- 
structed to  measure  visual  cQmauntoatlon  skills^  Some 
areas  of  visual  literacy  are  more  familiar  to  educators 
than  others.     Scores  tend  to  be  lower  when  subjects  are 
unfamiliar  with  visual  literacy  or  visual  aommunlcatlon 
skills.     It  seems  evident  that  Interest  In  visual  literacy 
will  Increase,     This  is  suggested  by  the  number  of  persons 
asking  to  use  the  test  and  comments  about  It,     Finally , 
there  Is  a  relationship  between  visual  literacy  expertise 
and  media  education.     It  Is  suggested  that  the  Item  pool 
be  expanded  and  norffling  data  enlarged* 
Specifically,  results  suggeBt: 

1 ,  a  viable  visual  literacy  test  can  ba  construated 

2,  some  areas  of  visual  literacy  are  more  familiar 
to  educators  than  others, 

3,  SQores  tend  to  be  low  because  of  imf amlllarlty 
with  the  subject  of  visual  literacy  or  communication 

4,  test  reliability  could  he  increased  by  dividing 
the  test  into  subtests  with  the  major  categories  listed  in 
The  Table  of  Specif icatlons  as  guidas*    Reliability  could 
then  be  figured  for  each  subtest  instead  of  the  test  as  a 
whole , 

5,  reliability  would  probably  be  increased  by 


.er 


a 


giving  tlie  tsat  to  a  aon-spiclalizsd  group  with  a  wld 

ranse  of  individual  differenceB  with  respect  to  aadla. 

6.  iatereat  ia  visual  literacy  will  increase--.tW 
was  evident  from  coaiEeats  about  interest  in  media  and  the 
request  ior  permisaion  to  give  the  test  to  a  High  school 
Media  Studies  Class, 

7.  tnsrs  is  a  relatloaship  batweea  visual  literacy 
expertise  and  madia  eduoation. 
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Abstract 

The  application  of  research  and  theory  to  the  development  of 
instructional  materials  relies  largely  for  its  success  upon  testing 
laaterials  in  the  field  and  revising  thera  aceordlnglyp    This  need  not 
necessarily  be  a  compliGated  business*    The  practitioner  who  is  not 
a  trained  instructional  developer ,  can  test  the  effectiveness  5  feas-- 
ibility  and  the  affective  component  of  instruction*    This  paper  des=* 
crlbea  a  few  simple  methods  for  doing  so* 


In  1972  j»  AECT'b  Coraaittee  on  Definition  and  TerminDlogy  published 
a  daflnitlon  of  educacional  technology,  (1972) .     The  basis  of  this  dtf- 
inltion  was  a  atatement  by  John  Kenneth  Galbralths  from  Tkz  Hm  IndlMt^UaZ 
StaXz,  that  "Teclmology  is  the  systematic  appllcatlQn  of  scientific  or 
other  orgml^ed  knowledge  to  practical  tasks- ^   (1967,  p.  12).     The  three 
components  of  this  statement^  systematic  applleation^  organized  know= 
ledge j  and  practical  tasks,  describe  the  ralatlonship  that  exists  between 
research  on  the  one  hmd^  and  Iristrnctlonal  development  on  the  other, 
between  the  theory  and  the  practice  of  edncatlonal  technology.  AECt^s 
definition  can  be  interpreted  to  mean  that  educational  technology  Is  a 
threes-phase  problem-solving  process  whereby  educational  problems  are  '  ^ 
identified,  appropriate  kno^^ledge  is  marihalled  that  is  likely  to  provide 
a  solution  to  these  problems,  and  whereby  this  knowledge  is  systematically 
applied  so  that  the  solution  is  both  useable  and  effective*  Instructional 
development  is  pivotal  in  this  process * 

I£  we  look  a  little  more  closely  at  each  of  the  three  componarits 
of  Galbraith-s  statement,  the  nature  of  the  dcvclopineat  prQcess,  and  the 
relationships  between  the  various  roles  of  the  educationai  technologist 
become  clearer*    We  must  first  ask  what  ate  the  practical  tasks  of  ed- 
ucational technologists.    Within  the  limited  context  of  the  activities  of 
researchers  and  instructional  developerg,  the  answer  to  this  question  must 
be^     to  bring  about  situations  in  which  learning  can  effectively  occur* 
In  practice,  this  means  that  the  instructional  developer  is  told  of  in- 
structional need    that  has  to  be  met,    A  need  is  usually  taken  to  be  ''a 
measurable  discrepancy  between  what  is  and  what  should  b^"  (Kaufman^  1972)  | 
so  the  role  of  the  Instructional  developer  becomes  one  of  bridging  a  gap. 


with  theory j  research  and  iKperierice  as  his  bridging  materials.     To  take  a 
simple  example,  if  a  school  board  discovers  that  its  high-school  graduates 
cannot  readj  then  a  nead  has  been  identified  r     they  are  llliterejUe  (what 
is);  they  should  be  able  to  reiid  novelss  magazines,  and  newspapers  without 
difficulty  (what  should  be)  •     This  gap  is  measurable  insofar  as  such  f ac-- 
tors  as  working  vocabulary  size,  reading  speed  and  reading  habits  can  be 
assessed  empirically.     The  Instructional  developer's  probXemj  thens  Is  to 
develop  instructional  materials,  strategies,  environments  which  make  the 
need  disappear- 

One  place  the  instructional  developer  can  turn  for  a  solution  to 
problems  is  to  the  body  of  research  and  theory  that  has  been  accumulated 
over  the  years.     Here  may  be  found  i-nformation  about  how  developers  solved 
similar  problems  elsewhere ^  which  can  be  applied  directly,  or  more  general 
psychological  theories  of  how  people  readj  why  students  can  get  through 
school  and  still  be  illiterate ^  and  the  best  ways    of  teaching  reading, 
IhB  developer  must  also  look  at  the  type  of  learner  in  question.    Are  they 
motivated?    Are  they  of  high,  average  or  low  ability?    The  feasibility  of 
the  various  strategies  that  begin  to  emerge  as  possible  ways  of  solving  the 
problem  must  also  be  taken  into  accounts     The  developer ^  in  effect ^  Is 
gathering  a  large  amoimt  of  Information  about  possible  solutions  to  the 
problem  and  about  the  particular  context  in  which  the  problem  has  presented 
itself.     This  information  gatherings  through  learner  analysis,  task  analysis, 
feasibility  analysis,  and  the  provision  of  several  potantial  solutions  * 
is  a  critical  step.     Solutions  to  instructional  problems  can  never  be 
b€>tter  than  the  Information  on  which  they  were  based,  and  they  are 
frequently  worse. 

It  is  in  the  systematic  application  of  this  information  to  a 
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particular  problem  that  the  instructional  developer  must  excel •     The  ap- 
plication of  theory  Co  piactlcal  problems  Is  systematic  if  it  meets  two 
criteria:     firsts  the  theory,  experience  and  research  that  are  applied 
must  be  pertinenti  seconds  the  decisions  arising  from  the  application  of 
particular  research  and  theory  must  be  empirically  tested*    The  second 
point  is  extremely  Important •     It  does  not  matter  how  "correct''  or  rel-- 
evant  rasearch  and  theory  appear  to  be.     I'hey  will  never  be  perfectly 
appropriate  to  the  particular  circumstances  in  which  the  instructional 
developer  has  to  apply  an  instructional  solution.    After  all,  tests  of 
statistical  significance  are  never  made  at  the  100  percent  level.    And  the 
generalizabillty i  that  characterizes  theory i  Implies  the  need  for  adaptation 
to  specific  situations*     So  however  sure  the  Instructidnal  developer  is 
that  the  right  instructional  decisions  have  been  mades  the  instructional 
products  must  nevertheless  be  tried  out  in  the  field  and  revised  until 
predeterained  standards  are  achieved  at  which  point  the  developer  will  be 
sure  that  the  need  has  been  met.     The  rest  of  this  paper  addresses  Itself 
to  this  question* 

Evaluation 

Before  getting  into  the  "how  to"  of  field  testing,  it  Is  necessary 
to  dwell  for  a  moment  on  the  more  general  question  of  evaluation.     It  is 
customary  to  distinguish  between  two  general  types  of  evaluation;  formative 
and  summative  (Bloorn,  Hastings  and  MadauSj  1973,^'  The  difference  between 
the  two  is  described  by  Gagne  and  Briggs  (1974)  in  teras  of  what  each  type 
is  used  for.    In  their  words,  formative  evaluation  is  used  to  "form'*  in- 
struction §  summative  evaluation  is  used  to  determine  the  "sunmed"  effects 
of  instruction  on  learners.     The  main  purpose  of  formative  evaluation  is 
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to  '*debug"  early  veriions  of  instructional  materials  and  activities,  to 
see  if  they  work,  to  identify  weaknesses ^  to  find  out  why  they  do  not  work, 
with  a  view  to  using  the  infomatiori  gathered  by  avaluation  to  revise  the 
materials  and  activities  in  %^ays  that  correct  those  problems*  Information 
producad  by  formative  evaluation  is  prescriptive.     The  purpose  of  suimatlve 
evaluation  is  to  gather  information  that  is  descriptive.    Summatlve  eval- 
uation is  carried  out  on  larger  units  of  inatruction  that  are  in  some 
sense  complete.     The  information  gathered  telli  us  how  the  finished  package 
performs  in  normal  instructional  settings ,  and  is  often  used  to  validate 
instruction  (Thlagarajans  1971).    This  paper  is  concerned  with  formative 
evaluation* 

So  that  it  is  useful  to  the  instructional  developer  (this  term 
applies  to  anyone  who  is  developing  instructional  matariala ^  not  necessarily 
a  profeisional  developer) ^  formative  evaluation  of  instruction  must  provide 
information  on  the  ef fectiveness *  feasibility  and  affective  qualities  of 
the  instruction  under  development*    While  there  is  a  tendency  for  field 
testing  to  get  complicated^  each  of  these  three  kinds  of  information  can 
be  gathered  easily.    Yet  even  simple  methods  must  allow  the  developer  to 
do  more  than  simply  conclude  that  the  Instruction  does  not  workj  cannot  be 
iTOplementedj  or  that  the  teachers  and  students  do  not  like  it.    The  assess^ 
ment  of  the  effectiveness  of  instruction  under  development  must  not  only 
evaluate  how  much  students  learn,  but  must  also  permit  the  developer  to 
pinpoint  what  the  student  does  not  learn  well,  and  why*    The  purpose  of 
formative  evaluation 5  remember ^  is  to  guide  revision  of  materials.     It  is 
therefore  important  to  know  not  Just  that  revisions  are  needed ,  but  where 
they  are  needed.     Likewisei  formative  evaluation  must  identify  any  reasons 
for  Infeaaibility  of  Implementation.     If  teachers  say  that  they  cannot 


manage  to  carry  out  their  part  in  the  proposed  instructional  process,  the 
developer  needs  to  know  why  so  that  feasible  alternatives  can  be  tried 
neKt  time  around.     Similarly ^  the  affective  component  of  formative  aval-- 
uatJon  must  determine  why  students  like  soma  parts  and  not  others ^  which 
parts  are  boring  and  which  are  exciting^  which  are  interesting  and  which 
dull.     Only  then  can  the  portions  learners  like  less  be  made  more  palatable  * 

Evaluation  in  each  of  these  three  areas  is  only  practical  if  a 
level  of  acceptable  performance  for  the  instruction  is  detemined  before'- 
hancL     After  all,  even  the  best  designed  Instruction  can  never  be  one 
hundred  percent  perfect.     It  would  therefore  be  a  waste  of  the  developer's 
time  to  keep  on  revising  materials  until  all  students  scored  full  marks j 
or  all  teachers  could  take  part|  or  everyone  liked  them*     It  la  safe  to 
say  that  the  earliest  revisions  are  the  easiest  to  make  since  they  deal 
with  the  most  obvious  errors  and  consequently  will  bring  about  the  most 
marked  improvements-     Each  subsequent  round  of  field  testing  and  revision 
will  require  increasingly  finer  adjustments  to  be  made  tmtil  a  point  is 
reached  where  the  improvement  in  instruction  brought  about  by  the  revision 
is  so  small  that  it  is  not  to  be  worth  the  effort  (and  cost)  to  make  it* 
While  the  developer  can  usually  tell  when  this  point  has  been  reached j  it 
is  of  considerable  benefit  to  try  to  anticipate  when  the  point  of  balance 
between  what  is  effective  and  what  is  worthwhile  will  be  arrived  at.  This 
is  done  by  setting  up  criteria  for  acceptability.     For  eKample,  it  might 
be  decided  that  no  further  revisions  will  be  carried  out  when  all  learners 
score  eighty  percent  or  more  on  a  posttest^  or  when  e  simple  majority  of 
teachers  judge  the  new  instruction  to  be  feasible,  or  when  no  more  than 
ten  percent  of  the  learners  say  that  they  do  not  like  what  has  been  developed. 
Sometimes  these  criteria  are  set  by  the  developer's  client,  as  in  the  case 


of  performance  CQacractlng.     But  this  need  not  concern  us  now.     Suffice  it 
to  say  that  criteria,  set  beforehand^  often  help  the  developer  make  the 
decision  that  further  revisions  are  not  necessary. 

The  rest  of  this  paper  describes  a  few  simple  yet  perfectly  adequate 
techniques  for  f leld^testlng  instructional  materials*     They  are  not  in- 
tended for  fully-fledged  instructional  developers s  who  have  their  own 
repertoire  of  far  more  sophisticated  tests  and  methods.     They  are  intended^ 
rather^  for  the  eduational  practitioners  who  is  not  a  trained  or  experienced 
developer j  who  has  neither  the  iime  nor  the  money  to  carry  out  elaborate 
evaluation  studies ^  and  whose  development  actlvitiea  are  modest  in  scope* 
The  methods  to  be  described  have  been  gathered  from  far  and  wide,  and  for 
the  most  part  used  successfully  by  staff  in  the  Learning  Technology  Unit 
at  the  University  of  Calgary  (Learoing  Technology  Units  1979)^ 

Field  Testing  Instructional  Effectiveness 

Criteria* -  Typical  criteria  for  the  acceptance  of  instruction  arei 
i*     Percentage  performance  on  a  posttestj  ranging  from  a  low  of 
70%  to  a  high  of  90%  depending  on  the  task  or  content, 
ii.    Number  of  students  making  errors,  which  is  particularly  useful 
for  debugging  instruction  in  motor  skills- 
ill.     Time  taken  for  students  to  complete  a  task  on  which  100% 

success  is  expected ,  suchj  again ,  as  performing  simple  motor 
skills s  or  straightforward  recall* 
Iv,    Percentage  of  students  ''passing"  a  posttest,  where  the  pass 
mark  could  be  100% j  or  could  vary  as  in  i* 
V*    Estimates  of  creativity,  originality ,  success  at  problem-solving 
for  higher-level  leamlng  (Gagnl,  1970*  Bloom,  1956). 
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Task  DasGrlption.     The  developer  must  Ideritify  what  discreet  tasks 
the  learner  must  be  able  to  perform  in  order  to  achieve  overall  succasa.. 
^is  breaking  down  of  instruction  into  Its  sub-parts  must  be  carried  out 
for  all  types  of  instruction.     It  is  wrong  to  think  of  the  word  "task"  as 
applying  just  to  motor  skills*     Cognitive  content  can  be  broken  domi  into 
Intellectual  tasks.     The  task  description  pravldes  both  a  framework  around 
which  the  developer  builds  instructional  materials  and  activities,  and 
also  a  chack^list  that  is  used  in  f ield^testingp 

For  psychomotor  skills ,  a  task  description  can  be  carried  out  by 
thinking  through  the  task,  ^d  writing  down  on  a  sheet  of  paper  every 
step  that  has  to  be  completed ^     It  is  best  to  er^  on  the  side  of  being  too 
detailed  at  this  point*     It  is  easier  to  remove  steps  from  the  list  than 
to  add  them  later.    Better  still  is  to  have  a  person  who  can  do  the  task 
go  through  it  J  while  writing  down  everything  that  the  person  does.  This 
way  it  ±s  less  easy  to  overlook  steps  when  you  think  it  through.  For 
cognltiva  skills s  working  through  the  task  yourself  and  consulting  others 
are  about  the  best  you  can  do  to  compile  your  list  of  sub^tasks.  You 
cannot  see  what  is  going  on  inside  someone  elsei  head  (though  this  can  be 
deduced  fairly  accurately  in  some  instances  5  such  as  solving  mathematical 
problems,  or  translating  into  a  foreign  language j  since  each  step  or  word 
produces  something  on  paper) *     More  complicated  mid  empirical  methods 
exist  for  the  malysls  of  cognitive  tasks;  but  they  afe  beat  left  alone  by 
all  but  the  most  highly  trained  developers* 

The  result  of  the  task  description  is  a  list  of  smaller  tasks  in 
sequence.    These  sub^-taska  form  the  basis  for  the  evaluation  of  the 
effoctiveness  of  Instructlori. 


Testing.     To  test  the  effectiveness  of  Instruction  :.n  motor  skills  ^ 
the  developer  must  actually  observe  the  leanier  working  through  the  series 
of  tasks j  and  note  on  a  score  sheet  whether  each  step  is  successfully  cora^ 
pleted  or  not.     Learners  should  work  through  the  instructional  material 
and  associated  activities s  and  should  then  be  tested  individually  on  the 
skill.     All  that  it  is  necessary  to  record  is  whether  the  learner  was 
successful  or  not  in  completing  the  task*     (A  typical  checklist  for  test- 
ing a  motor  skill  ™  laminating  a  picture  "  is  givan  in  figure  one)* 


Figure  1  About  Here 

If  the  learner  becomes  completely  stuck ^  the  examiner  can  help  so  that 
the  learner  can  move  onto  the  next  step  In  the  sequence.    A  note  should 
be  made  of  the  eKaminer* s  Intervention. 

Testing  cognitive  skills  will  usually  involve  a  written  test  to  be 
completed  after  instruction.     Remember j  this  is  formative j  not  summative 
evaluation.     It  is  therefore  necessary  to  test  not  just  the  cumulative 
effects  of  instruction  on  the  learner's  knowledge  and  ability ^  but  the 
successful  acquisition  of  each  aub^part  of  the  total  skill  as  it  is  en^ 
countered  along  the  way.    As  a  general  rulej  there  should  therefore  be  at 
least  one  question  on  each  cognitive  sub^task  identified  in  the  task 
description.     Fox  eKample,  if  the  objective  of  the  instructional  unit  is 
for  the  learner  to  be  able  to  describe  why  an  airplane  flies,  and  the  task 
deacription  has  identified  the  understanding  of  vector  diagrams  and  of 
airflow  over  a  curved  surface  as  tasks  leading  to  an  understanding  of  the 
whole  process 5  then  the  learner  should  be  asked  at  least  one  question  on 
vector  diagrams  and  one  on  airflow  as  well  as  general  questions  on  how 
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planes  fly.     The  reasoni  for  this  is  that  if  you  just  had  a  learner  dc^scribe 
how  planes  fly^  and  the  learner  was  unable  to  do  so,  than  you  would  not 
know  if  it  %mu  because  there  was  a  lack  of  utiderBtandlng  of  vector  diagrams 
or  of  airflow.    You  would  know  that  instruction  had  not  been  successful, 
but  you  would  not  know  where  the  learner  had  had  difficulty. 

Tests  of  cognltiva  abilities  may  be  of  any  kind  —  multiple  choice, 
short  answer,  or  even  essay^  provided  that  Instructions  are  given  to  the 
learner  which  require  an  eKplanatlon  of  all  pertinent  sub^taskSp    A  word 
of  warning,  though.     It  Is  difficult  to  construct  good  multiple-choice 
tests,  which  are  valid ^  reliable  and  discriminate  well  between  those  who 
have  learned  the  material  and  thos^  who  have  not. 

Sample  Size,    You  are  field-testing,  not  conducting  a  sophisticated 
psychological  study.     The  early  rounds  of  field-testing  should  identify 
the  most  serious  weaknesses  and  the  most  obvious  mlatakes*    These  should 
become  apparent  in  the  behavior  and  In  the  written  ans%^ers  of  the  first 
few  of  your  learners*    On  some  psycho-motor  ekllls,  it  Is  not  necessary 
to  test  more  than  three  or  four  learners  in  the  first  round  of  field--teat- 
Ings  Incf easing  the  number  to  half  a  dozen  or  as  many  as  eight  in  subsequent 
roimds.     The  same  is  true  for  cognitive  skills.    Written  answers  from  four 
learners  wtll  give  you  a  satisfactory  first  estimate  of  the  succeaa  of 
your  Instructional  materials.    Keeping  the  numbers  small  will  also  leave 
sufficient  learners  in  the  pool  of  potential  "subjects'*  for  subsequent 
rounds  of  testing  and  evaluation. 

Interpreting  Scores.    There  Is  no  need  to  submit  learners-  scores 
to  elaborate  statistical  analysis*    Looking  at  individual  learners'  scores 
is  quite  sufficient  to  identify  where  problema  lie.     Figure  two  shows  a 
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scorG-^sheet  of  errors  committed  by  Just  two  learners  during  field  tescing 


FigurQ  2  About  H©re 

of  the  laminating  unit  mentioned  above.    Each  learner  made  two  attempts 
at  the  task*    Both  missed  step  eight  the  first  time  around  Ccutting  off 
excess  laminating  film) ,  and  one  of  them  became  quite  confused  over  the 
whole  sequence  for  preparing  the  film  to  cover  the  illustration  during 
the  second  trial.     This  ahould  be  enough  to  indicate  to  the  developer  that 
the  Instruction  was  not  clear  about  removing  excess  film  and  preparing 
the  film  to  lay  over  the  lllustrationi     Testing  another  fifty  learners 
would  not  add  much  more  information  about  this. 

The  Interpretation  of  scores  on  written  answers  of  whatever  kind 
works  in  the  same  way.     Incorrect  answers  to  questions  based  on  sub-skills 
upon  which  an  understanding  of  the  whole  la  based  point  to  where  the 
instruction  is  not  as  effactive  as  it  might  be.    The  Inability  of  more 
than  one  learner  out  of  four  to  answer  correctly  a  question  on  vector 
diagrams  would  suggest  to  the  developer  of  the  unit  on  flight  that  in-- 
struct  ion  was  weak  at  this  point* 

Revision.    The  developer  now  looks  at  the  faulty  sections  of  the 
Instructional  materials  that  have  been  identified  in  the  testing.  Reviatons 
can  be  referred  back  to       any  point  in  the  development  process*  Perhaps 
the  section  of  videotape  showing  eKCiss  laminating  film  being  cut  off  was 
terribly  out  of  focus,  and  Just  needs  to  be  re-^shot*    Perhaps  the  producar 
did  not  think  It  was  terribly  Importimt,  and  showed  it  far  too  quickly 
to  be  imderstood.    Perhaps  it  is  not  really  necessary  to  cut  off  excess 
film  at  this  point.    Or^  perhaps  the  understanding  of  vector  diagrams  is 


asking  too  much  of  the  learners,  and  the  objecCive  is  not  attainable. 
Whatever  revlsioris  are  needed,  the  developer  should  make  them,  and  submit 
the  materials  and  activities  to  a  second  round  of  testing.     The  test^ 
revise  cycle  is  coritlnued  until  the  pre-established  criteria  have  been 
met . 

Evaluating  the  Feaalbllity  of  Instruction 

Criteria.     Typical  criteria  for  assessing  the  feasibility  of  lm= 
plamenting  instruct lonal  materials  and  activities  are: 

i.     Cost.     Cm  the  "client-'  afford  to  purchase  and  maintain  the 
instructional  system? 
il.    Available  equipment  and  facilities.    Normally,  cost  and  availabl 
facilities  are  known  before  development  begins  5  so  these  con-- 
strainti  would  be  taken  into  account  before  f ield^testlng* 
iii.    Personnal.    Are  the  teachers  who  vill  be  using  the  instructional 
materiala  and  activities  trained  to  use  them? 
iv,    Cost--e£fectlveness.    Maybe  learning  takes  place  and  the  client 
can  afford  the  system.    But  it  is  still  relatively  eKpensiva 
for  the  amount  of  learning  that  takes  place. 
V.    Difficulties  In  carrying  out  instruction.    Maybe  the  Instruction 
is  successful,  but  requires  an  inordinate  amoimt  of  tlmaj  or 
preparation. 

Evaluation,     The  evaluation  of  the  toetructional  materials  and 
activities  against  these  criteria  can  be  done  informally.    Cost  and 
facility  factors  cm  be  assessed  quite  simply  by  questioning  teachers  and 
administrators.    Personnel  skills  and  difficulties  in  carrying  out  the 
instruction  can  be  estimated  by  direct  observation  of  teachers  and  learners 


trying  the  materials  out  in  classrooins*    Maybe  teachers  need  Inservlce 
training  to  implement  the  instruction^  and  maybe  the  activities  and  mat-- 
erlals  should  be  organised  in  differaat  ways  so  that  they  are  easier  to 
carry  out  and  use*     The  observer  In  the  classroom  should  take  notes  of 
any  logistical  or  other  problems  that  occur  so  that  modifications  can  be 
made  to  overcome  them* 

The  above  s trasses  the  importarice  of  doing  at  least  some  of  your 
field  testing  in  classroom  settings  ±f  it  is  at  all  possible*    tho.  testing 
of  individual  learners ,  described  In  the  previous  sectionj  needs  to  be 
eupplemented  by  tryouts  under  conditions  as  close  as  possible  to  those 
in  which  instruction  will  be  implemeated*     It  la  only  through  this  kind 
of  observation  that  certain  problems  can  be  detected*    A  set  of  self- 
instructional  unltSs  produced  by  the  Learning  Technology  Unit  at  the 
University  of  Calgary  (Learning.  Techriology  Units  1979)  Tm  into  a  problem 
which  observers  noted*    Materials  designad  for  use  by  one  learner  were  in 
fact  being  worked  through  by  several  learners  at  a  time*    As  a  result  of 

i 

this  a  division  of  labor  occurred  smi  no  one  student  performed  all  of  the 
required  tasks*     The  reason  for  this  tos  a  higher  enrolment  in  a  lab, 
coraponeat  of  a  course  than  had  been  ratlcipated,  and  the  problem  was  over'- 
come  by  duplicating  stations  in  the  lab,  area*    The  infeaslbllity  of 
self ^instruction  under  these  circumstances  might  have  gone  undetected  had 
not  part  of  the  f ield'-testing  involved  direct  observation  In  the  class-- 
room  setting. 

Evaluating  jhe^  Affective  Component 

Criteria*    Instruction  should  reach  accaptable  levels  in  the 
following  areas i 


i.     "nie  students  should  like  the  activities  and  materials •  If 
twenty  percent  or  more  indicate  dislike ,  revisions  should  be 
made* 

±±.     ThB  materials  should  capture  and  hold  the  interest  of  the 
learners*     If  twenty  percent  or  more  indicate  boredoms  the 
ttaterials  should  be  made  more  interesting, 
iii*     The  materials  should  provide  encouragement  Cfeadbackj  rain-- 

forcement)  to  the  students*    If  thay  feel  a  lack  of  encourage-' 
TOent  or  of  direction^  revisions  should  be  made* 
Testing  Affect,    More  is  going  to  be  said  in  this  session  about 
measuring  attitudes.     So  let  us  look  at  a  few  simple  ways  of  finding  out 
learners*  affective  reactions  to  instruction,.    Urle  Bronf anbrenner  (1976) 
has  poirited  put  that  the  simplest  way  to  find  out  If  learners  like  the 
instruction  j  (or  eKperlmental  treatments)  they  receive  is  to  ask  theLi* 
Every  developer  should  at  least  do  this.    As  learners  rim  through  the 
Instructional  materials  and  are  tested,  it  is  not  difficult  to  ask  them  a 
few  questions,  such  as*     ■■Did  you  find  this  interesting  or  boring?"  "t^as 
it  too  easy  or  too  difficult?-^     '-^ould  you  like  to  be  taught  more  lessons 
this  way?'-    Responses  to  these  informal  questions  can  be  noted  for  future 
reference*     Interest         be  Increased  in  revisions  by  speeding  up,  or  even 
just  changing  the  pace  of  instruction |  by  switching  from  a  verbal  to  a 
visual  form  of  presentation ,  or  vice  versa  I  by  Inserting  an  example  into 
an  eKposltory  segment  of  instruction,  and  so  on. 

Further  refinements  to  these  simple  techniques  can  be  made*  Infor-^ 
matloti  can  be  obtained  by  having  each  learner^  after  working  through  the 
materials >  rate  statements  on  Likert  scales*    Typical  statements  would 
bei     "I  foimd  the  material  interesting/*  rated  from  "strongly  agree**  to 
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"strongly  disagree".     Information  about  the  learner 'a  likes  and  dislikes 
cari.  111  this  way,  be  quantified,  and  soraetinies  this  is  an  advantage.  Or 
an  affective  component  can  be  built  into  an  observer's  checklist 
(ihiagarajans  1971) •    Herej  in  addition  to  a  checkmark  or  a  cross  against 
each  step,  to  Indicate  success  or  failure,  other  symbols  can  be  used.  An 
exclamation  point  might  Indicate  boredom^  an  asterisks  enthusiasm.  In 
this  way,  revisions  am  be  directed  to  specific  portions  of  the  instructional 
materials  and  activities,  and  motivational  devices  can  br  built  into  the 
parts  of  the  materials  where  they  are  most  needed. 

It  is  also  important  to  teat  the  affective  reactions  of  the  teachers, 
or  of  whoever  it  is  who  will  be  present  when  the  materials  are  used.  The 
students  may  leam  what  they  are  supposed  to^  and  they  may  Ilka  the  mat- 
erials very  much*    But  if  the  teachers  do  not  approve  of  the  way  the 
learaers  are  to  be  taught,  or  If  they  are  not  enthusiastic  about  what  the 
developer  has  produced,  then  it  is  likely  that  success  will  be  difficult 
to  achieve*     Again,  teachers  can  simply  be  asked  their  opinion,  or  can 
rate  statements  on  scales.     The  developer  needs  no  more  information  than 
this , 

jummary 

A  Minimum,    For  someone  who  is  not  aGcustomed  to  doing  in= 

structional  development,  and  who  is  faced  with  the  task  of  developing  and 
producing  materials  and  activities  for  a  particular  purpose,  the  fore-- 
going  has  provided  some  suggestions  as  to  how  the  developer's  product  can 
be  tried  out.     The  purpose  of  that  trial  is  to  provide  information  that 
will  permit  sensible  revisions  to  be  made  to  materials  and  activities  with 
a  minimum  o£  fuss.     To  achieve  this,  it  is  not  necessary  for  the  casual 


developer  to  carry  out  an  elaborate,  empirical,  statistical  evaulation. 
But  the  developer  should  at  least  carry  out  the  follgwlngi 
1*     Set  criteria  for  accepCable  learner  performance. 
11.     Determine  criteria  for  feasibility  of  Implementation* 
ill.     Determine  what  will  be  an  acceptable  affective  rating, 
Iv,     Perform  a  task  degcriptlori. 

V.     Prepare  an  observer's  checklist,  or  written  test,  depending  on 
the  content,   (motor  skill  or  cQgnltive) ,  to  evaluate  the 
effectiveness  of  the  Instruction  for  each  step  as  well  as  for 
overall  ability. 

iv.    After  giving  the  test,  or  observing  performance,  ask  the  learners 
if  they  liked  the  lustruction,  if  they  found  it  hard  or  easy, 
vii.    Quiz  teachers  and  administrators  on  the  feasibility  of  ±m- 
plementing  the  instruction* 
vlii.    Ask  teachers  if  they  liked  the  instruction,  and  If  they  think 
it  is  appropriate  for  the  intended  learners. 
Ix.     Carry  out  revisions,  after  a  try-^out  with  four  to  six  learners, 
X,     Try  to  field'-test  the  instruction ,  or  some  part  of  it,  in  a 

real  classroom  setting. 
Useful  Elaborations.    For  those  with  a  little  more  time  and  In-- 
Itiatlve,  a  few  additional  features  can  be  added  to  f ield-'testlng  which 
make  the  task  easier,  more  empirical  and  more  objective  I 

xi.     If  you  have  skill  in  the  matter,  write  multiple^choice  tests 
so  that  the  dlstractors  perform  a  diagnostic  function,  i.e. 
b.  is  the  right  answer,  but  choslng  c.  Indicates  that  the 
learner  has  a  different  kind  of  difficulty  from  the  learners 
who  chose  a* 


o  b  i  u 
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xli*    Develop  a  coding  system  on  observation  checkliats  to  indicate 
reactions s  such  as  boredom  and  enthuslasmp  beyond  Just  right 
s^d  wrong, 

xiii.    Use  rating  scales  for  affective  responses. 

With  these  few  simple  tools,  any  teacher  or  anyone  who  is  not  a 
trained  instructional  developer,  should  be  able  to  carry  out  simple  fields- 
tasting  of  instruction  and  revise  it  accordingly*    l^hile  development  of 
large  and  cQmplex  instructional  systems  requires  far  more  elaborate  field-^ 
testings  and  while  even  field-testing  of  simpler  systems  and  materials 
can  be  more  complicated^  educational  technology  is  not  inherently  esoteric, 
Galbraith-s  definition  of  technology  as  the  application  of  knowledge  to 
practical  tasks  is  not  difficult  to  understand ^  nor  to  apply*    And  it  is  in 
the  applications  through  field-testing 5  of  theory  to  practice^  that 
educational  practitioners  can,  in  their  everyday  activities,  become  ed-- 
ucational  technologists. 
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OUTCOMES  OF  THE  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT  PROJECT 

Robert  K.  Branson 
Florida  State  University 

ABSTRACT 

This  project  sought  to  meet  documented  Army  training  needs  through 
application  of  system  analytic  procedures.    Doctrine,  regulations,  and 
school  organization  were  investigated.    Regulations  were  clear  and 
comprehensive.    School  organization  lacked  a  clear  training  evaluation 
component.    During  the  project,  the  Army  reorgamzed  the  schools  to 
reflect  requirements  of  the  regulations  and  to  permit  proper  execution 
of  the  new  training  doctrine.    This  change  greatly  increased  the  role 
of  evaluation  1n  the  schools.    Recommendations  were  made  about  organization 
and  regulations  and  recommended  doctrine  was  published  in  detail  as  the 
Interservice  Procedures  for  Instructional  Systems  Development.  Lessons 
learned  and  management  recommendations  are  presented. 


OUTCOMES  OF  THE  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT  PROJECT 

Robert  K,  Branson 
Florida  State  University 

OrientatiQn 

The  session  for  which  this  paper  was  prepared  had  as  the  central  them 
"Lessons  Learned"  in  major  Instructional  development  projects.  This 
paper  will  begin  with  the  broader  focus  of  human  resources  development, 
then  concentrate  on  instruction  as  one  of  the  major  approaches  to  human 
resources  development.    By  limiting  the  scope  principally  to  training 
programs^  it  will  be  possible  to  highlight  specific  issues  not  always 
apparent  in  typical  education  settings* 

The  distinction  between  education  and  training  is  made  here  simply 
on  the  basis  of  the  probability  that  the  position  or  job  classification 
following  training  is  known  prior  to  an  individuals  beginning  the  train- 
ing program.    For  example ^  consider  the  baccalaureates  in  chemistry. 
The  probability  of  a  good  prediction  of  their  future  employment  when  they 
enter  college  is  low.    At  the  other  extreme^  a  sailor  who  enters  the 
Navy's  radioman  school  has  a  high  probability  of  going  to  a  radioman's 
job.    Instruction,  then^  for  the  radioman  would  be  training^  for  the 
chemistry  majors  education.    Not  because  of  the  subject  matter  but  because 
of  the  predictability  of  the  job  for  which  the  individual  was  being 
prepared.    Obviously,  In  this  sense.  Instruction  for  which  there  was  no 
occupational  outcome  regardless  of  how  specific  would  be  cast  under  the 
rubric  of  education.    Dentistry  is  a  training  program,  business  is  ques- 
tionable, while  teachers  are  trained^  history  majors,  education,  etc. 


The  Project 

The  project  to  be  described  in  this  paper  was  undertaken  by  the 
Center  for  Educational  Technology  (CET)  at  Florida  State  University. 
The  Center  was  organized  by  Robert      Morgan ^  Robert  M.  Gagne,  Leslie 
J*  Briggs*  and  the  author  in  the  early  1970's  for  the  purpose  of  taking 
a  systems  analytic  approach  to  the  solution  of  human  resources  develop-- 
ment  problems  nationally  and  internationally.    Here,  a  systems  analytic 
approach  refers  to  the  application  of  psychological  and  human  factors 
research  and  development  techniques  to  the  solution  of  man-environment 
pe'rformance  problems.    Eacii  of  the  individuals  iioted  has  ditalled  his 
own  view  of  how  such  applications  can  best  be  made  by  presenting  either 
theoretical  treatises  or  the  description  of  large-scale  projects.  To 
gain  a  better  perspective  of  how  these  authors  view  the  analytic  approach, 
seet  for  example,  Gagna  1966,  1977;  Briggs,  1977|  Morgan  and  Chadwick^ 
1971;  Morgan  1979;  and  Branson,  et  al,,  1975. 

The  U,S.  Army  Combat  Arms  Training  Board  (CATB),  a  highly  effective 
and  creative  military  training  organization,  approached  the  Center  in 
1973  to  assist  them  in  making  a  major  revision  and  upgrading  of  the  approach 
to  Army  training*    One  request  made  of  CET  was  to  assess  the  state  of  the 
art  in  empirically  designed  training  technology,  to  identify  concepts 
appropriate  to  the  mission  of  Army  schools  and  training  centers  and  to 
recommend  methods  for  institutionalizing  these  concepts  within  the 
Army  school  and  training  center  system*    Careful  human  engineering  of 
the  organization  of  guidance  material  and  doctrine  was  essential,  part1» 
cularly  in  providing  "how  to  do  it"  guidance  to  military  and  civilian 
training  specialists  within  the  Army  school  system    (Branson,  Stone,  Hannum, 
and  Rayner,  1973). 


A  second  request  was  to  employ  appropriate  development  procedures 
to  prepare  and  validate  a  training  model  with  detailed  guidance  proce- 
dures to  develop  in  Army  personnel  the  skills  necessary  to  enable  them  * 
to  analyze^  design*  develop.  Implement  and  control  empirically  designed 
instruction*    It  was  my  privilege^  and  I  might  add,  a  considerable 
professional  challenget  to  have  been  selected  as  the  principal  Investi- 
gator on  that  project. 

The  CET  Approach 

Principally  because  of  the  orientation  of  the  founders  of  the  Center 

effective  projects i  a  somewhat  traditional  systems  oriented  human 
resources  develoDment  approach  was  taken,    A  brief  description  of  that 
approach  would  be  useful.    It  is  well  founded  in  the  psychological 
literature  that  there  are  nonrially  only  three  ways  to  achieve  or  improve 
the  performance  of  individuals  in  their  Jobs.    One  can  select  people  who 
already  know  how  to  do  a  job.    This  approach  is  commonly  taken  by 
commercial  airlines  when  they  hire  former  military  pilots  who  have  already 
logged  thousands  of  hours  in  multi-engine  service* 

A  second  approach  is  taken  through  the  provision  of  sufficient 
instruction  to  individuals  so  that  they  will  be  able  to  perform.    In  a 
military  context,  literally  thousands  of  people  are  trained  every  day  to 
repair  vehicles  and  aircraft^  to  troubleshoot  electronic  systems^  and  so 
on^  people  who  knew  nothing  about  those  systems  prior  to  their  entry 
into  the  military  service, 

A  third  approach  to  improving  or  achieving  performance  is  through 
effective  management  and  supervision.    Such  approaches  would  Include  the 
manipulation  of  economic  incentives  for  sales  people,  for  example,  or 


through  work  enrichment  proctdLiris  to  create  more  favorable  working 
conditions  and  assignments  for  people  on  the  job.    General ly*  business 
and  industrial  organizations  pay  far  more  attention  to  the  management 
aspects  of  performance  than  they  do  to  the  selection  and  training 
aspects  of  performance* 

Mager  and  Pipe  (1970)  have  detailed  these  three  approaches  in  a 
straightforward  fashion.    It  is  their  view  that  before  any  effective 
human  resources  change  can  be  made^  it  is  first  necessary  to  document 
a  clear  performance  discrepancy.    That  is,  to  define  an  outcome  gap 
between  what  Is  and  what  ought  to  be.    Some  of  those  outcome  gaps  will  be 
caused  by  faulty  selection  procedures,  others  will  be  caused  by  skill 
deficiencies,  while  others  will  be  caused  by  Inadequate  approaches  to 
rnanagement.    Training  can  only  be  a  solution  for  performance  discrepancies 
that  are  documented  to  be  skill  deficiences. 

It  is  important  to  recognize  that  all  human  resources  development 
programs  or  projects  will  be  effective  only  when  they  are  viewed  as  a 
means  to  the  achievement  of  some  known  outcome.    While  that  should  be 
obvious,  frequently  it  has  been  found  that  a  large  number  of  people  view 
training  ai  an  end  in  itself  rather  than  a  means  to  achieve  an  acceptable 
ieyei  of  performance.    There  is  abundant  literature  detailing  situations 
in  which  training  was  used  as  a  solution  to  a  non-training  problem,  and 
was  therefore  used  Ineffectively,    One  reason  for  the  apparent  failure 
of  effective  training  programs  is  that  they  represent  the  most  innocuous 
intervention  rnanagement  can  take. 

The  processes  of  documenting  specific  outcome  gaps  have  been  detailed 
by  Kaufman  (1976).    The  processes  of  needs  analysis  and  assessment  are 
undertaken  to  establish  program  validity.    In  a  training  program,  validity 


refers  to  the  correlation  between  job  requirements  and  the  instruction. 
An  Important  first  step  in  defining  outcome  gaps  is  the  Identification 
of  an  organization- s  mission.    An  organization  should  first  be  evaluated 
only  against  its  own  mission.    That  is  1mportant.,feetause  there  is  often 
disagreernent  among  outsiders  on  what  an  organization's  mission  ought  to 
be  or  should  be^  rather  than  what  it  actually  is.    Kaufman  also  argued 
that  it  1s  our  obligation  to  question  an  organization's  mission^  but^  once 
established,  the  mission  then  becomes  the  guidance  for  the  entire  organl- 
' zation. 

Once  the  organlEatlon's  fmssion  has  been  established  it  has  been 
rather  a  straightforward  matter  to  compare  the  operations  of  the 
organization  to  its  mission  including  the  way  that  the  enterprise  is 
organizedi  the  resources  available^  and  the  use  of  those  resources  in 
the  accomplishment  of  the  mission.    It  is  a  fundamental  requirement  that 
results  or  outcomes  be  an  important  part  of  the  total  assessment  procedure. 
Prior  history  has  shown  that  we  far  too  often  get  concerned  with  process- 
rather  than  outcome*    These  outcomes  or  results  must  then  be  compared 
to  plans  or  expectations.    Finally  the  quantitative  or  qualitative 
discrepancy  between  results  and  plans  becomes  the  operational  definition 
of  the  outcome  gap. 

The  Politics  of  Discrepancies 

To  a  seasoned  systems  analyst^  the  discovery  and  documentation  of  an 
outcome  gap  represents  an  opportunity  to  apply  a  variety  of  solution 
alternatives.    Unfortunately,  to  the  manager  of  an  agency  or  a  corporate 
executive,  the  documentation  of  an  outcome  gap  often  becomes  a  political 
problem  of  considerable  proportion.    On  the  one  hand  it  1s  impossible  to 
go  forward  with  precision  in  the  absence  of  documented  outcome  gaps.  On 


the  other  hand  the  eKistence  of  a  documented  outcome  gap  often  requires 
action  on  the  part  of  the  executives  responsible.    These  actions  are  not 
always  solution  oriented. 

When  the  documentation  of  an  outcome  gap  is  done  for  the  purpose  of 
establishing  a  political  advantage^  it  is  often  the  eKicutive's  tactic 
to  discredit  those  who  prepared  the  documentation.    For  example^  in 
February*  1979,  the  General  Accounting  Office  issued  a  highly  critical 
report  of  Peace  Corps  recruiting  activities,  citing  as  evidence  that  the 
Peace  Corps  had  "failed  to  interview"  adequately  a  large  number  of  those 
hired.    This  "discrepancy"  was  cited  as  an  example  of  poor  management  on 
the  part  of  the  Peace  Corps.    Yet,  fifty  years  of  psychological  research 
on  interviewing  has  demonstrated  that  the  interview  1s  the  least  effective 
commonly  employed  method  of  employee  selection, 

A  second  approach  taken  to  the  handling  of  discrepancies  is  often  that 
of  finding  someone  to  be  the  scapegoat.    Whose  fault  is  it  that  the 
training  program  is  not  working?    Let's  find  the  people  responsible  and 
replace  or  discipline  them*    This  decisive  executive  action  creates  the 
illusion  that  the  problem  has  somehow  been  solved  when  in  fact  it  will 
recur*    Having  effectively  scapegoated  the  problem,  however,  the  executive 
can  remain  in  place  until  the  ultimate  tragedy  occurs,    Scapegoating  often 
protects  Individual  jobs  but  rarely  contributes  to  mission. 

Generating  Solution  Alternatives 

Sometimes  it  Is  possible  to  document  discrepancies  in  an  organization 
for  which  no  one  can  reasonably  be  scapegoated.    The  responsibility  for  the 
problem  Is  so  diffuse  that  no  one  can  be  clearly  blamed.    It  was  the 


fortunate  case  1n  the  military  ISD  project  to  have  been  able  to  document 
the  discrepancy  in  such  a  way  that  making  it  public  did  not  require  adminis= 
trative  or  punitive  action-    Once  opportunities  of  that  kind  are  discovered 
it  is  possible  to  begin  constructive  deliberations  on  solution  alternatives. 

Locating  Contributors 

It  was  decided  early  in  the  military  Instructional  Systems  Developinerit 
(ISD)  project  to  encourage  wide  participation  as  a  means  of  generating 
a  base  of  participation  and  support.    Firsts  we  visited  several  Arfny 
Installations  to  identify  knowledgeable  and  effective  staff  memebers.  Each 
of  the  Army's  schools  and  training  centers  has  a  number  of  civniaris  and 
fnilttary  personnel  who  are  competent  in  the  applications  of  a  s^stenis 
approach  to  training  (SAT)-    The  participation  of  these  specialists  in 
the  development  of  the  ISD  materials  was  critical.    First,  they  have 
e;<perienced  a  wide  range  of  prior  training  research  and  developinGnt 
activities*    Seconds  they  could  serve  as  a  responsive  editorial  board  for  the 
critique  of  any  tentatively  selected  approach.    Many  of  them  could  make 
particularly  good  estimates  of  what  will  and  what  will  not  work  in  various 
environments. 

Selected  people  from  those  instanations  visited  were  invited  to  attend 
various  planning  and  briefing  sessions  during  the  planning  and  design  of 
the  project.    Accumulating  a  group  of  experts  or  specialists  to  address  a 
problem  is  not  without  risks.    Many  participants  had  already  made  up  their 
minds  about  "the"  correct  approach  before  they  arrived,    FortLinately*  others 
adopted  a  problem  solving  orientation  and  were  willing  to  consider  a 
variety  of  solution  alternatives.    Those  who  had  strongly  entrenched  views- 
whose  views  did  not  Prevail --either  became  detractors  from  the  project  or 
dropped  out  all  together.    Perhaps  sonie  day  we'll  be  able  to  mrk  out 


an  effective  method  to  get  such  groups  to  reiTiain  in  a  problem  solving  mode 
rather  than  a  solution  selling  mode. 

It  should  be  noted  here  that  the  Army  has  worked  out  over  the  years 
a  variety  of  methods  for  handling  decision  making*    OtiG  of  those  and 
probably  the  most  prevalent  is  that  of  having  the  senior  officer  present 
make  the  fifial  decision  on  a  project  or  program.    Until  the  timp  that  thp 
decision  is  inade,  people  are  free  to  argue,  to  present  al ternativeSj  to 
oppose  others  and  so  forth*    Hovvevers  after  the  decision  is  made  It  is 
expected  that  all  others  participating  will  execute  the  action  plan  vnth 
diligence.    But*  even  in  the  Army,  the  better  decisions  are  more  faithfully 
executed  than  those  which  are  suboptlmal. 

The  Army  State- of" the-Art  in  1973 

In  addition  to  identifying  the  key  people  who  were  at  the  various 
training  centers  and  schools  in  the  Arniy,  a  second  purpose  of  our  visits 
to  those  Installations  was  to  define  the  existing  state  of  the  art. 
Because  the  Army  had  done  the  early  basic  research  in  the  systems  approach  to 
training,  it  v/as  decided  that  a  clear  picture  of  exactly  where  they  were 
in  1973  would  be  basic  to  niaking  any  real  Improvements.    Accordingly ^  m 
analyzed  the  training,  training  literature,  school  organization,  and 
training  of  trainers  status  as  of  the  first  quarter  of  1  973. 

Three  Important  areas  were  analyzed  and  then  evalijated  in  terms  of 
the  mission  they  were  intended  to  accomplish.    The  first  of  these  areas 
was  that  of  finding  out  what  the  people  were  supposed  to  do  and  how  they 
were  supposed  to  do  it.    In  the  Army  what  people  are  supposed  to  do  1s 
spelled  out  in  regulations,  and  how  they  are  supposed  to  do  it  is  spelled 
out  in  other  forms  of  guidance*    The  regulation  which  specified  what  and 
how  the  Army  was  to  train  in  1973  v^as  called  the  Systems  Engineering  of 
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Training,  Regulation  That  regulation,  issued  in  1968i  had  as 

its  basis  a  fairly  standard  systems  approach  to  training  model.  That  model 
is  displayed  in  Figure  1. 

A  second  consideration  was  how  people  were  prepared  to  do  what  they 
were  supposed  to  do.    Because  there  were  a  large  number  of  people  in  the 
Army  school  system  concerned  with  training  and  an  even  larger  number  of 
people  attending  Army  schools,  there  must  be  considerable  and  regular 
attention  paid  to  how  people  are  prepared  to  their  jobs.    Thirds  after  peopl 
have  been  prepared  to  do  their  jobs  then  it  is  necessary  to  have  some 
regular  and  objective  way  of  finding  out  how  well  they  did  their  job. 

The  results  of  the  prelTminary  studies  and  visits  left  us  with  a 
good  understanding  of  what  the  Army  schools  were  supposed  to  do^  how  the 
people  at  the  schools  were  prepared  to  do  their  work,  and  how  they  were 
evaluated  and  Informed  of  the  quality  of  their  work.    The  last  element 
of  that  analysis  was  that  of  understanding  how  they  were  organized  to 
accomplish  the  assigned  work. 

In  the  Systems  Engineering  of  Training  model  displayed  in  Figure  1, 
the  first  step  or  function  called  for  Is  that  of  job  analysis.    It  follows 
according  to  management  logic  that  if  job  analysis  is  a  function  required 
to  be  performed  then  someone  should  be  assigned  to  perform  that  function. 
Further,  we  should  be  able  to  find  on  the  organizational  chart  where  in 
the  organization  tfiat  that  function  Is  performed.    In  this  sense  the 
organizational  chart  is  a  road  map  which  allows  us  to  match  function  to 
organization* 

Displayed  in  Figure  2  is  the  organizational  chart  that  was  presented 
to  us  by  schools  we  visited  during  1973.    We  were  able  to  track  the  function 
of  job  analysis  down  through  the  Deputy  Conimandant  for  toctrine  through 
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the  Directorate  of  Training  Development  and  found  that  organization  charged 
with  the  job  analysis  function. 

Subsequently',  we  tracked  each  of  the  other  functions  specified  in 
the  regulation  down  through  the  organizational  chart  until  we  could  find 
the  person  who  was  actually  in  charge  of  conducting  the  work  specified  in 
the  regulatipn.    Each  time  we  found  a  discrepancy  between  what  was  called 
for  in  the  regulation  and  what  was  actually  being  done,  either  organiza- 
tionally or  functionally  within  the  school,  we  documented  that  discrepancy. 

Putting[  the  -ing  in  Training 

Fundaniental  to  the  theory  of  a  systems  approach  is  the  notion  of 
control .    Control  refers  to  the  process  of  planning,  executing  the  plan, 
obtaining  results,  comparing  the  resuTts  to  the  plan,  and  acting  on  the 
discrepancies.    Because  there  are  many  elements  of  a  training  system, 
identifying  the  method  of  control  becomes  a  key  aspect  of  analyzing  the 
effectiveness  of  the  organization. 

Consider  as  an  example  an  accounting  department.    In  the  accounting 
department,  accountants  apply  standard  procedures  and  techniques  to  each  of 
the  accounts  on  a  periodic  basis  and  present  periodic  reports.    The  purpose 
and  form  of  those  reports  is  specified  in  advance  and  the  person  In  charge 
of  accounting  makes  certain  that  each  of  the  accountants  applies  the 
specified  procedures  in  a  craftsnianlike  manner.    Individual  accountants  are 
not  free  to  handle  their  accounts  according  to  their  own  whim.  Further, 
any  deviations  from  standard  practice  are  documented  and  explained.  Not 
only  are  accounts  managed,  and  accountants  managed,  but  the  entire  process 
of  accounting.  1s  managed  to  meet  specifications  and,  accordingly,  is  con- 
trolled. 


Applying  that  model  to  training  would  require  that  specified  plans 
be  made  for  the  trainees,  that  the  trainers  would  conduct  training 
according  to  specification  and,  that  the  results  of  training  would  be 
compared  to  those  planss  discrepancies  documented^  and  action  initiated 
on  those  discrepancies. 

In  the  Army  schooVs  model,  we  were  unable  to  Identify  any  evidence 
that  training  as  such  was  being  controlled.    Trainers  were  being  trained 
and  trainees  were  being  trained  by  trainerf,  but  the  results  of  that 
training  were  not  compared  to  the  plan  by  anyone  specifically  charged 
with  that  responsibility.    While  It  may  be  true  that  people  in  various 
departrnents  had  interest  in  certain  operations  data  about  training*  no  one 
was  charged  with  the  responsibility  for  docurnenting  training  results. 

Based  on  this  analysis  of  the  regulation  and  the  organization  of  the 
schools^  we  documented  this  major  organizational  deficiency  in  a  consultants 
report  to  the  president  of  CATB  in  1973* 

Our  interviews  with  training  specialists  in  the  field  identified  many 
who  claimed  that  the  regulation  under  which  they  were  operating  was  not 
a  good  regulation.    A  consultant  groups  HumRROj  was  hired  to  investigate  the 
application  of  the  regulation  at  the  Army  aviation  school.    While  that 
report  contained  many  useful  recommendations  which  were  followed  in  the 
ISD  project  regarding  ways  to  improve  the  regulation  itself  and  Its 
documentation  and  evaluation  procedures^  no  mention  was  made  of  the  fact 
that  the  school  was  not  organized  to  Implement  the  regulation  (Ricketsont 
Schulz  &  Wright,  1970). 

One  viewpoint  from  which  the  ISD  project  analysis  was  made  included 
the  following  basic  tenant:    If  a  function,  result,  task,  or  outcome  is 
everyone's  responslbil ity»  it  will  not  get  done*    Applying  this  rule  to  the 


Army's  1973  school  organization^  the  evaluation  and  control  of  training 
was  widely  dispersed  in  the  organization  and  as  a  consequence  did  not 
occur  in  a  meaningful  and  deliberate  way. 

While  there  are  no  "laws"  which  apply  universally  in  training 
management,  and  there  is  no  such  thing  as  the  correct  organization,  a 

often  than  net.  We  have  generally  approached  new  situations  with  these 
principles  in  mind.    The  first  six  laws  of  training  management: 

1.  Systems,  models,  designs,  and  equipment  do  not  achieve 
results.    Managers  do. 

2.  If  a  function,  result,  task,  or  outcome  is  everyone's  respon- 
sibility, it  will  not  get  done. 

Corollary:    A  manager  can  only  handle  N-1  collateral  duties. 
(One  less  than  the  number  already  assigned.) 

3.  Any  activity  which  Is  not  thoroughly  checked,  or  inspected 
according  to  specification  or  requirement  will  not  be  done 
according  to  specification  or  requirement. 

4.  Incumbents  in  any  job  (mechanics  or  instructors)  do  not  want 
to  learn  how  to  do  It  cur  way--they  want- to  do  it  their  way, 
no  matter  how  historicany  ineffective. 

5.  An  enterprise  must  be  adequately  organized  and  resourced  to 
accomplish  mission.  : 

Coronary:  Training  and  education  are  resource  sinks;  no 
quantity  of  resources  will  ever  be  perceived  as  adequate, 

6.  To  be  effective,  a  management  systems  analyst  must  pick  a 
problem  to  work  on  which  is  big  enough  to  be  important  but 
for  which  no  one  can  reasonably  be  blamed. 


Our  conclusion  in  1973  was  that  the  regulation  was  perfectly  adequate 
for  the  intended  purpose.    There  was  insufficient  guidance,  examples,  and 
training  provided  for  people  to  follow  the  regulation  but  those  are 
peripheral  issues  to  the  question  of  the  quality  of  the  regulation.  Thus, 
if  the  regulation  has  all  of  the  attributes  of  an  SAT  model,  there  is  little 
point  1n  revising.    What  was  required  was  detailed  guidances  training^  and 
manager  training. 

At  that  point  in  the  projects  we  tried  to  summarize  the  data  that  had 
been  gathered  so  that  we  could  test  the  quality  of  our  inferences  about 
the  status  quo*    It  was  important  that  these  inferences  be  as  accurate 
as  possible  since  they  would  serve  as  the  baisls  for  all  future  plans. 

In  that  context,  it  occurred  to  us  that  placing  the  blame  on  the 
regulation  was  a  clever  rationalization  on  the  part  of  the  managers.  Perhaps 
not  even  a  conscious  one.    What  we  found  was,  to  us,  a  clear  violation  of 
the  first  law  of  training  management:    Systems,  procedures,  models ,  and 
machines  do  not  produre  reulsts;  managers  do. 

The  violation  was  not  that  the  managers  were  not  working  hard  or 
that  they  were  ignoring  the  duties,  it  was  that  no  specific  training 
results  were  required  and  no  one  was  responsible  for  achieving  (or  evaluating) 
those  results.    Results  in  this  context  refers  solely  to  outcomes^  not 
processes.    The  Army  was  certainly  diligent  in  applying  the  training  process. 

An  effective  solution  would  require  not  only  a  means  for  specifying 
and  measuring  outcomes^  but  also,  an  organizational  recognition  that  a 
senior  manager  must  be  held  accountable  for  the  achievement  of  those 
outcomes.    To  achieve  an  organizational  recognition  of  those  outcomes 
would  require  a  major  reorganization  of  the  schools.    That  reorganization 
was  clearly  beyond  the  scope  of  work  for  which  we  had  been  contracted^ 


and  more  importantly^  well  be^'ond  the  decision  authority  of  our  sponsoring 
agency.    Consequently,  we  were  not  optlnilstic  about  a  fundamental  change. 

The  Development  Plan 

By  early  summer  of  1973  we  had  completed  the  analysis  of  the  state-of- 
the-art  in  training  1n  the  Army  (Task  I)  ends  with  the  assistance  of  a 
number  of  training  experts,  had  been  able  to  outTine  the  project  plan. 

This  plan  Included  the  deyelopment  of  an  Army  training  model  (to  replace  the 
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one  in  the  systems  engineering  regulation)^  development  of  a  detailed  set 
of  procedures  to  provide  training  guidance  to  users,  technical  level 
workshops,  workshops  for  the  immediate  supervisors  of  technical  work^  and 
workshops  for  senior  managers.    The  workshops  for  the  supervisors  and 
managers  were  thought  to  be  the  most  critical  feature  of  the  project.  This 
management  emphasis  was  based  on  the  assumption  that  supervisors  cannot 
readily  achieve  results  they  do  not  understand. 

In  addition  to  the  specifications  for  completing  the  project,  a 
methodology  was  established  1n  which  each  of  the  major  program  elements 
would  be  developed,  critlqueds  and  thoroughly  staffed  before  proceeding 
to  the  next  step.    While  this  process  of  printing  and  distributing  draft 
materials  was  time  consuming,  it  was  absolutely  essential.    The  process 
of  asslmiliatlng  and  interpreting  the  critique  and  suggestions  was  difficult 
at  best.    Often 5  we  found  ourselves  torn  between  two  equally  effective 
altGrnative  approachess  being  able  to  use  only  one  of  them  for  consistency 
purposes*    Finally^  we  projected  a  requirement  to  use  the  materials  for  a 
period  of  18  months^  visit  Installations  where  they  were  being  used, 
conduct  an  in-depth  evaluation  of  the  use  of  the  materials,  and  then  do  a 
final  version  based  on  the  formative  data  accumulated  during  the  tryout 
of  the  draft.    Unfortunately,  that  revision  did  not  occur. 


It  is  difficult  to  be  sure  why  the  revision  did  not  take  place 
according  to  schedule.    Some  of  the  reasons  centered  around  the  change 
in  project  officer  and  a  change  in  organization  of  CATB.    There  was  also 
a  small  group  of  powerful  civilians  called  '^education  advisers 5"  whose 
jobs  we  had  recommended  to  be  eliminated  or  restructured  during  the 
Task  I  study j  and  there  was  a  change  1n    TRADOC  at  the  highest  levels. 
Perhaps  the  contractor  was  simply  ineffective  in  convincing  the  Army 
that  the  final  version  was  necessary--they  still  issue  the  draft  version. 

Mission  Drift 

Hardly  a  month  had  passed  following  the  award  of  the  contract  for 
the  development  of  an  Army  model  and  set  of  Army  procedures  when  we 
encountered  an  organization  called  the  Interservice  Training  Review 
Organization  (ITRO)^    A  subconimittee  of  that  organization  was  at  the 
time  attempting  to  standardize  training  terminology  among  the  services. 
For  those  of  you  in  the  university  community^  this  task  was  akin  to 
zhat  of  trying  to  get  several  schools  within  a  state  university  system 
to  agree  on  common  courses.    It  always  sounds  so  logical  on  the  face  of 
it  until  one  actually  tries  to  execute  a  plan. 

Upon  hearing  of  our  project^  that  ITRO  committee  approached  the 
Army  with  the  proposal  that  the  model  and  procedures  being  developed  for 
them  be  converted  to  an  interservice  sponsorships  that  is,  the  project 
would  be  supervised  by  respresentative  members  of  the  Army,  Navy^  Air 
Force  and  Marine  Corps.    In  the  interest  of  interservice  harmony,  the 
Army  agreed  to  that  proposal. 


While  it  seems  reasonable  that  it  ought  to  be  possible  to  prepare 
training  materials  which  would  be  useful  to  all  members  of  the  armed 
forces,  that  reasonableness  applies  only  to  principles^  rules,  generali- 
zationsj  concepts  and  the  like.    It  does  not  work  at  all  when  one  attempts 
to  describe  training  problems^  training  proceduresj  sources  and  uses  of 
udtd,  drid  other  service  specific  requirernents.    Clearly,  our  mission  had 
begun  to  drift.    It  is    "ten  difficult  to  please  ev^eryone.  Fortunately^ 
at  that  times  the  project  manager  assigned  by  the  Army  remained  the  same. 
As  all  defense  contractors  are  awares  high  quality  projects  cannot  be 
completed  if  high  quality  project  managers  are  not  assigned  to  complete 
them. 

It  is  the  duty  of  the  project  manager  to  keep  the  contractor  clearly 
oriented  toward  the  mission  and  objectives.    Such  a  responsibility  is 
particularly  important  when  it  is  necessary  to  work  with  a  large  number  of 
people  in  order  to  complete  the  project.    Oftens  working  with  a  large 
number  of  people  permits  the  introduction  of  noise  into  the  system. 
The  government  project  manager  must  taken  into  account  the  legitimate 
suggestions  and  concerns  of  those  participating  while p  at  the  same  timsa 
keeping  the  project  on  track  so  that  the  outcome  will  be  in  their  best 
interests. 

Late  in  the  project^  we  went  through  the  rather  shocking  experience 
of  a  change  of  project  manager.    The  first  project  manager  was  a  highly 
rated  lieutenant  colonel  in  the  regular  Army.    When  he  was  reassigned 
overseas,  he  was  replaced  by  anenergetic^  technically  competent  captain 
in  the  Utah  National  Guard.     Imagine  your  al l-^American  high  school 
quarterback  suddenly  being  sent  to  the  Dallas  Cowboys.    Captains^  no 
matter  how  brillant,  do  not  vote  as  much  stock  as  lieutenant  colonels. 


We  believe  the  lesson  learned  here  is  that  it  is  not  in  the  best 
interest  of  the  contractor  to  change  project  managers  in  the  middle 
of  the  project  unless  one  can  make  a  substantial  change  for  the  better. 
We  believe  this  lesson  would  generalize  across  a  wide  range  of  projects. 

Results 

The  outcomes  of  the  project  can  be  described  in  two  major  categories 
Those  which  were  expected  and  those  which  were  not.    The  major  expected 
outcome  of  the  project  was  the  publication  of  all  of  the  draft  materials 
developed.    These  materials  included: 

0      Interservlce  ISD  Model  (See  Figure  3) 

0      The  manuals  published  in  the  Army  as  TRADOC  Pam  3^0-30  and 
in  the  Navy  as  NAVEDTRA  106A 

0      The  workshop  materials^  including  the  exercises  and  the  work^ 
shop  director's  guide 

0      A  slide/tape  presentation  which  provides  an  overview  of  the 
entire  set  of  procedures. 

All  of  the  materials  described  above  are  referenced  at  the  end  of 
this  paper  with  their  document  acquisition  numbers  so  that  interested 
civilians  can  find  them  accessable. 

A  significant  element  in  the  Task  I  problem  analysis  was  that  of 
the  organisational  structure  of  the  Army  school  system.    Perhaps  the 
most  dramatic  unexpected  outcome  during  the  entire  project  was  the 
reorganization  of  the  Army  school  system.    Following  the  publication  of 
the  manuals  (in  late  1975)5  the  Army  issued  a  draft  organizational  chart 
for  the  school  systems  in  1976,    This  revised  organization  is  referred 
to  as  School  Model  '76.  is  presented  as  Figure  4*    Of  particular  interest, 
in  Figure  4  is  the  Directorate  of  Evaluation  which  has  been  placed  on 
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Figure  4.    U.S.  Army  School  Model  1976 


an  equal  level  with  all  other  major  divisions  of  the  organiEation.  What 
this  means  is  that  the  Directorate  of  Evaluation,  which  previously  did 
not  exist,  now  is  on  the  same  level  as  the  Directorate  of  Training 
Development,  the  Directorate  of  Training,  and  the  Directorate  of  Combat 
Development,    Now,  the  people  who  develop  the  content  of  training  work 
in  a  different  department  than  those  who  develop  the  training  and  tralnirg 
materials.    Those  who  conduct  the  training  are  in  a  different  department 
from  those  who  evaluate  the  outcomes  of  training.    This  independence 
of  major  function  provides  a  system  of  checks  and  balances  which  permits 
feedback  to  be  channeled  to  appropriate  elements  of  the  organization 
as  a  means  of  improving  outcomes,  and  the  potential  for  gaining  control. 

Figure  5  shows  how  the  relationship  between  the  organizational 
departments  relates  to  the  phases  of  the  ISD  model.    The  Directorate  of 
Training  Development  is  responsible  for  Phases  I,  II,  and  III  of  the  ISD 
Model.    The  Tra'^ning  Directorate  is  responsible  for  Phase  IV  of  the  ISO 
Model.    And  Phase  V,  Evaluation,  is  placed  In  the  Evaluation  Directorate. 
The  School  Nodal  '76  represents  a  considerable  conceptual  improvement  over 
the  prior  organization- since  it  placed  equal  emphasis  on  the  evaluation 
of  outcomes  as  it  does  with  the  conduct  of  training  processes.    If  this 
training  organization  can  be  kept  in  place  long  enough  for  its  systematic 
attributes  to  mature  and  operate,  it  will  have  been  a  major  contribution 

to  school  management. 

A  second  unexpected  outcome  was  that  the  final  version  of  the  materials 
was  not  completed.    A  thorough  revision  was  planned  based  on  broad  and 
sufficient  usage  data  in  the  Army.    The  revision  was  to  have  been  similar 
to  a  revisiori  of  instructional  materials  based  on  formative  evaluation 
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data.    As  a  consequence^  many  rough  edges  (and  perhaps  some  huge  potholes) 
remain  in  the  materials.    Such  revisions  cannot  be  competently  done  by 
people  not  thoroughly  familiar  with  the  principles  upon  which  the 
procedures  are  based. 

What  we  feari  and  there  Is  evidence  to  support  this  fear.  Is 
that  the  potholes  and  rough  edges  win       identified  by  some  users. 
These  users  will  try  to  remove  the  pu^^tiois  of  the  doctrine  offensive 
tn  them,  even  though  that  section  or  step  was  a  fundamental  part  of  the 
principles  being  established.    If  this  practice  continues^  the  integrity 
of  the  entire  model  can  be  degraded. 

The  third  unexpected  outcome  was  the  degree  uf  r^sfstance  among 
several  civilian  education  advisers  within  the  school  system.    One  the 
one  hands  they  were  unable  to  muster  really  significant  support  for  their 
resistance,  but  on  the  other  hand^  they  did  create  a  substantial  block 
to  implementation.    Fortunately^  their  protest  was  ill-timed.    Had  they 
put  forth  their  maximum  effort  at  the  time  the  project  director  was 
changed,  or  at  other  times  of  partial  confusion,  they  could  have  increased 
their  chances  to  prevent  the  publication  of  the  materials. 
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7  h  i  3  p  e  m  m  n  t  b  t  i  o  n  will  f  i  r  s  t  a  o  in  pare?  a  n  d  c:?  o  n  t  r  a  s  t  the 
i\  i  ffBvi  n  ^Ji  p  I !  r  p  Q  ii>  e  iH  a  f  1  r  i  s  t  f  ^  *  j  a  t  i  a  n  b  I  m  nd  e  v  a  1  u  ^3 1  :L  v  ©  e  o  fti  p  u  t  e  b  tii  s  e  d 
%  %  ill  u  1  B  t  :L  a  n  s  ♦      1 1     w  :l  11  o  n  t  u  d  i  ^  c  u  ts  the     cj  e  v  ^  1  o  m  e  r*  t     of  & 

a B  r d  1  a  1  a f^sj  %i  m u  I  b  t  i  o n  t h a t  will  be?  u b e d  & ii>  m  v^f^ve n t  for 
dimcussian*  The?  dt^si^n  mnd  mtructurB  of  this  r©fe?rerit  simulation 
will     th^n  pvB^mntBd  *     The  maJor  part  ion  of  thim     paper  will 

discuiiis  the  (svaluation  dtsBi^^n  of  this  card ialo^y  mimulstion  and 
mamB  !Yie<iM:iU  reiTiGnt  iiiisuefs  of  cMDiTiFuter  baMff^d  evaluative  siiTiulat  iDn  * 


The      technique    of    computar    based    simulation    has  been 
successfully    utilized  in  an  instructional  fflofle  for  sone  time  in 

many     academic    disciplines*     Within  the  past  two    decades^  much 

prelinainary    work    has  been  conducted  to  investigate  the    use  of 

simulation      techniques,       on      and     off     line*       as  evaluative 

instruments.     While    the    technigue  of  constructing     a     mpael  of 

reality     for  simulation  purposes  remains  essentially  the  same  for 

either      an    evaluative    or    an    instructional     slfflulationy  the 

instructional    aeslgns     differ    vastly.     This    paper     will  first 

discuss     the  differences  in  instructional  design  necessitated  by 

the  differing  educational  purposes  of  these  two  types  of  computer 

based    simulations.     Once  these  differences  are  established^  than 

development  and  evaluation  issues  will  be  presented  and  discussed 

using    an     ongoing    research  project  at  the     Onlversity    of  Iowa 

College    of  f!edlclna  as  a  model  reference*     the  final  portion  of 

this    paper    will  discuss  consiaeratlons  relevaat  to    scoring  of 

evaluative      simulations      anfl      the     Issues    of      validity  and 

reliability . 

Ea£EaJ§s^of^Instructioaal^and^^ 

The  purpose  of  an  instructional  computer  based  simulation  is 
to  allow  the  student  to  Int^vract  with  a  siiculated  model  and  to 
learn  how  this  simulated  enviornment  reacts  to  differing 
conditions  imposed  on  it.  Most  Instructional  sinulatlons  have  as 
part  of  their  design  some  Instructional  segments^  or,  programs 
that  help  to  guide  and  teach  the  learner  various  principles  or 
conditions  present  in  this  simulated  enylornraent.  Many  of  these 
instructional  segments  are  tutorial  in  that  they  teach  the 
student  some  of  the  salient  features  of  the  model,  or  simulation. 


and  allow  the  learner  to  ptogress  from  very  basic  principles^  ^  or 
concepts,  to  more  cDapleX  aspects  of  the  system  of  interest.  The 
more  sophisticated  insttuc tional  simulations  utilize  remedial 
branching  as  part  of  their  design  to  reinforce  If^rning.  Some  of 
these  cofflplex  instructional  slfflulations  have  internal  test  itess 
that  are  usefl  as  a  check  for  learner  comprehension  anfl 
understanding  of  the  concepts  presented  as  part  of  the  tutorial 
package- 
Once  the  learner  has  successfully  grasped  the  basic 
principles  and  concepts,  s/he  is  first  taught  how  to  interact 
with  the  model,  or  simulated  environment #  and  ultimately  allowed 
to  experiment  and  manipulate  this  model.  In  this  phase  of  the 
instructional  simulation  the  learner  sets  up  the  varying 
conditions  and  constraints  and  then  explores  how  the  model  reacts 
given  these  determinants. 

To  summarize,  the  general  design  of  instructional  computer 
based  simulations  is  to  first  create  a  model  of  the  object 
system,  to  develop  a  series  of  instructional  tutorial  programs 
which  teach  various  concepts  and  principles  of  the  object  system 
and  to  ultimately  allow  the  learner  to  interact  with  this  model 
by  manipulateing  and  varying  the  conditions.  The  primary  purpose 
of  this  type  of  computer  based  simulation  is  instructional* 

By  contrast,  the  primary  purpose  of  an  evaluative  simulation 
is  to  test,  or  measure  Knowledge,  skills^  and/or  competencies 
within  a  specific  discipline  or  subject  area.  Its  purpose  is  not 
instructional;  any  learning  that  does  occur  is  secondary  to  the 
main     purpose     of  evaluation.     The  same  principle  holds     here  as 


with  anj  written  CDgnitlve  test  instrument:  whatever  a  stiidtnt 
learns  while  being  tested  is  purely  secondary  to  the  purpose  of 
evaluating  what  a  student  "knows"  or  "understands'* . 

B e V e Igpnient  of  a  CardlpXogy  Evaluative  S i fi ul a t i g^n 

As  stated  earlier^  the  evaluative  simulation  which  will  be 
used  as  a  reference  is  one  developed  at  the  Untversitj  of  Iowa 
College  of  Medicine  in  Cardiolojy,  The  purpose  of  this  computer 
based  evaluative  slmiilation  is  to  assess  both  diagnostic  and 
therapeutic  skills  of  prloiary  care  physlctans  and  medical 
studGnts  on  acute  myocardial  infarction.  The  design  of  the 
simulation  allows  for  assessment  of  diagnDStic  skills  of 
physicians  in  sections  of  medical  history  taking,  physical 
examination  and  in  the  ordering  of  diagnostic  laboratory  tests. 
Patient  management  skills  are  evaluated  for  patients  who  have 
uncofflplicated  illness  ^  in  this  case  myocardial  infarction#  as 
well  as  in  section  dealing  with  common  complications  of  the 
disease;  ventricular  arrhythmias/  bEadycardia#  hypertension , 
cardiogenic  shocks  and  cardiac  arrest  and  resusciatlon * 

In  June  of  1977  a  development  team  consisting  of  a  content 
expert  {cardiologist}^  a  meflical  liason  (physician's  assistant 
and/or  a  senior  medical  student) ,  and  an  instructional  designer^ 
prograoiner  began  a  two  year  project  to  develop  an  on  line 
evaluative  simulation  in  carfliology,  A  review  of  the  literature 
was  first  done  to  ascertain  the  state  of  the  art*  Once  this  was 
accomplished/  the  content  expert  and  the  instructional  designer 
met     to  begin  the  content  development  of  this     simulation.  These 


"brain  storming"  sessions  lead  to  an  overall  design  of  ^the 
"sifflulated    pa  ot  enFironfflent,  Mhich  was     then  initially 

£ 7,0¥chart^i.^  this  point  a  senior  ffleaical  student  ^as  brousfht 

in  and  iai troflucea  to  the  concept  and  technique  of  evaluative 
simulations  and  flowcharting.  For  the  first  six  months,  aany 
meeting  occured  between  the  three  members  of  the  developnient  team 
involving  discussion  of  the  casa  and  the  ramifications  of 
diagnosis  and  treatment.  The  medical  men  had  no  compttter  or 
instuctional  design  background  and  the  instructional  designer  had 
no  medical  baclcground^  so^  each  member  of  the, team  learned  what 
exact  information  was  necessary  for  the  develDpfflent  of  each  part 
of  the  simulation. 

Since  the  major  portion  of  this  preseritation  is  to  center 
around  measurement  considerations  of  evaluative  simulations,  the 
design  and  development  aspects  will  not  be  detailed.  Suffice  it 
to  say  that  the  desigii^  dev?  T \7^3Dt ,  programming^  and  the 
debugging  processes  of  the  si^*^/ .\  ci^^.  ^  took  one  and  a  half  years 
to  accomplish.  The  programffling  was  done  in  BASIC  on  a  Hewlett- 
Packard  2000  Access  minicotDputer  located  at  the  University  of 
Iowa  Computer  Assisted  Instruction  Laboratory, 

fiSSiSLD«.Snfl_Struc  ture^gf  _thg^Ca 

Prior  to  consiaeration  of  some  EDeasurement  issues,  a  short 
discussion  of  doctor  (user)  interaction  with  the  simulation  model 
is  warranted.  Since  the  purpose  of  this  instrument  is 
evaluative,  tho  user  is  essentially  allowed  to  progress  through 
the     simulation     at  her/his  own  rate  and  with    ainimal  structure 


osed.  The  program  was  designed  to  alloi/  the  doctor  to  follow 
whichever  path(s)  s/he  chooses  and  is  conseguently  forced  to  deal 
with  whatever  complications  arise.  Thereforet  therQ  is  no 
direction  or  guidance  iro posed , 

There    is     no  instructional  refflediation  incorporated  within 
this     simulation^     since  instruction  is  not  the     intent     of  this 
instrument.     The     feadback    the  doctor  receives  is     twofold:  one 
form     is  the  result (s)    of  treatment (s)   that  were  ordered  and  how 
this     trea tment (s)    affected  the  patients'  condition*     The  purpose 
of      this       feedback    is     primarily    informational     rather  than 
instructional.     The  second  form  of  feedback  is  both  inforniational 
and  instructional.     This  feedback  occurs  either  at  the  end  of  the 
simulation  when  the  patient  is  successfully  treated  and  released 
from     the     hospital,     or,  if  during  the     simulation     the  doctor^ 
through     incorrect    diagnosis  or  treatment  kills    the  "patient". 
The     program  records  which  sections  the  doctor  has  completed  and 
once        the        simulation      is      terminated,       successfully  or 
unsuccessfully,     gives  detailed  feedback  in  terms    of  diagnosis, 
recommended     treatment  and  rationale.     This  feedback  is  included 
as  a  summary  of  the  case. 

The  structura  of  the  simulation  is  as  follows:  the 
simulation  is  broken  down  into  an  introductory  office  scenario^ 
and  a  series  of  nanagement  programs  on  premature  ventricular 
contractons  (PVC) ,  bradycardia  (day  2) ,  hypertension  (day  3),  and 
a  final  program  on  cardiogenic  shock  (day  ^)  ,  There  are  also  two 
different  cardio  pulmonary  resuscition  (CPh)  programs;  one  for 
management    of    cardiac  arrest  in  the  coranary  care  unit  and  the 


other  for  treatment  of  catdiac  arrest  in  an  office  or  unmonitpred 
ward.  There  is  CDntinulty  of  presentation  of  these  different 
management  pcograns ,  The  doctor  will  proceea  through  the 
simulation  without  realizing  that  theiss  programs  const  i  tuts 
different  sections* 

The  actaaX  logistics  of  this  seguencirig  will  not  be 
addressed  in  detail  as  it  is  complex  and  not  the  primary  intent 
of  this  paper.  It  is  iniportant  to  note  that  at  many  points 
throughout  the  simulation  the  patient  can  die  due  t,o  iiaproper 
managenientp  Consegtuently^  some  of  the  doctors  will  not  get  to 
all  the  sections  of  this  simulaiton  if  the  patieiit  dies 
prematurely. 

7alidity  Issues  of  .Computer  Based  Eyaluativa  Siaiulatlon 

With  the  purpose  and  structure  of  this  slmiilation  in  mind, 
attention  is  now  tutned  to  various  measurement  considerations  and 
the  research  design  that  will  be  used  in  evaluating  this 
simulation*  It  should  be  noted  that  the  measurement  Issues  that 
will  be  raised  ar%  not  specific  to  this  particular  simulation/ 
but^     generalize  to  most  evaluative  simulations  on  and  off  line. 

The  first  step  In  the  evaluation  prpcess  is  Falidation* 
Therefore^  the  primary  purpose  of  the  research  flesign  is  to 
validate  the  simulation.  The  audiences  that  will  bt:  used  are  not 
theDiselves  being  avaluated.  It  is  their  performances  pn  the 
simulation  and  on  various  estarnal  validation  instruments  that 
are  of  utmost  concern  In  ascertaining  tfc e  validity  and  the 
reliability  of  this  simulation. 


EKLC 


There  are  four  different  audiences  that  will  be  used  to 
validate  this  evaluative  simulation.  These  audiences  consist  of 
a  ^roup  of  senior  isdical  students  at  the  University  of  lowa^  a 
group  of  primary  care  physicians  who  are  returning  to  the 
nniversity  of  Iowa  for  a  five  day  workshop  as  part  of  a 
continaing  medical  education  program ^  the  staff  cardiologists  at 
the  University^  and  20  to  30  internists  located  throughout  the 
state  of  Iowa,  h  general  overview  of  the  validating  instruments 
will  be  discussed  as  well  as  th'  research  desigQS  for  each  group. 
The  data  handling  and  statistical  analysis  will  not  be  presented 
since  they  are  technical  rather  than  conceptual  issues. 

There  will  be  three  external  validating  in  uments  used:  I) 
a  cognitive  written  multiple  choice  exam  2)  a  subjective  clinical 
rating  scale  and  3)   a  medical  audit* 

The  cognitive  written  multiple  choice  exam  will  be  used  with 
the  senior  medical  student  group  and  the  group  of  primary  care 
physicians.  This  axa  will  contain  questions  which  address  the 
doctors*  knowledge  level  of  cardiology  and  cardiovascular 
disease.  There  will  also  be  a  series  of  guestions  used  which 
address  more  skill  related  areas  of  cardiolDgy.  These  guestions 
will  force  the  doctDr  to  go  beyond  the  knowledge  levels  to  apply 
and  analyze  specific  circumstances  in  'order  to  diaynose  and 
manage  patients  in  various  situations. 

This  exam  will  be  tailored  to  correspond  to  the  major 
management  points  addressed  in  the  simulation.  By  constructing 
the  e%am  in  this  manner^  it  will  enable  direct  comparison  of 
specific     performances     on     the  simulation  and     on     related  test 


items.  It  is  felt  that  this  is  the  only  waans  of  using  a 
cognitive  exam  as  a  validating  instrument.  To  help  validate  the 
simulation^  the  scores  obtained  on  the  simulatiDn  will  ho 
compared  with  the  exam  results  to  assess  1)  if  there  is 
relationship  between  scores  on  each  part  Df  the  exam  and 
performance  on  the  corresponding  segments  of  the  simulation  and 
2)  if  the  results  on  the  total  simulation  breakdown  into  the  same 
relative  ranlcings  as  they  did  on  the  exam. 

The  subjective  clinical  rating  scale  will  only  be  used  with 
the  medical  student  group.  The  major  problems  in  using  this 
subjective  scale  as  a  validating  inchrumant  are  inter  and  intra 
rater  reliabilities^  as  well  as  the  actual  observation  time  each 
rater  has  with  each  student.  There  is  unfortunitly  no  way  to 
regulate  the  amount  of  time  a  rater  has  with  each  student,  as 
most  of  these  env:...  ^  ers  are  spontaneous.  *he  consistancy  with 
which     a     rater  r   -    ^    .ndivldual  studfents  problem     with  any 

observational  instrument  and  an  attempt  will  be  ma\^  to  control 
for  this.  This  instrument  will  be  constructed  In  a  manner  to 
enable  the  students  to  be  rated  on  specific  diagnosis  and 
s^angement  criterion  that  corresponds  to  major  diagnostic  and 
management  points  in  the  simulation. 

The  ultimate  criterion  for  validating  the  simulation  is  to 
assess  how  well  a.tions  and  decisions  taken  on  the  simulation 
match  actions  taken  in  rea  1  life.  There  exists  in  medicine  a 
means  by  which  this  assessment  can  be  accomplished^  that  is  by 
conducting  a  medical  audit.  A  patients'  chart  contains  all  the 
relevant     informatiDn  on  the  diagnosis  and  management  of  a  case. 


It  is  with  this  record  of  information  that  a  case  can  be 
reconstructea  and  the  major  medical  decisions  determined,  A 
medical  audit  is  the  means  by  which  a  case  is  recoristructed. 

Twenty  to  thirty  internists  throiighou t  the  state  of  low  a 
will  be  participating  in  this  phase  of  the  validation*  With 
their  permission  an  audit  will  be  conducted  on  those  patients' 
charts  that  directly  relate  to  the  content  of  th^^  simalation. 
Specific  comparisons  will  be  made  between  performancG  in  real 
life^  as  assessed  through  the  audits  and  the  diagnosis  and 
management  of  the  simulated  patient* 


The  criterion  related^  or  raore  specifically^  the  cQncarrent 
validity  of  this  simulation  will  be  assessed  by  using  the 
cognitive  writton  test  instrument.  The  relative  uankings  on  the 
oxam  will  be  comparad  with  the  relative  rankings  for  both  groups 
on  the  siEulation. 

It  is  assumed  that  this  written  cognitive  instrument  is  both 
valid  and  reliable  and  is  in  fact  measuring  the  same  skills  that 
the  simulation  is  neasuring.  This  is  a  critical  assuEnption.  A 
simulation  does  not  duplicate  reality^  it  only  imitates  it, 
Becaus^^  of  this^  in  assessing  the  concurrent  validity  of  this 
Simula h.^ ':.n  it  is  only  possible  to  determine  some  aspects  of 
physicMn  performance,  This^  or  any  simulation  can  only  give  a 
measu  or  ho^  a  physician  is  capable  of  behaving  not  how  well 
s/he  k^ili  h''iadvB  in  the  actual  situation.  It  is  important  to 
bear  this  in  mind  Khen  determining  the  concurrent  validity  of 
simulations. 

The  clinic  il  ratings  of  the  student  group  will  also  be  used 
to  assess  concuriant  validity.  This  too  will  use  relata^/e 
rankings  of  the  students  on  individual  sections  of  the?  clinical 
rating  scale.  The  results  on  specific  sections  of  the  simulation 
will  b.  compared  with  the  ratings.  It  is  hoped  that  there  vfill 
prove  to  be  a  relationship  between  how  the  studerits  perform  in 
the  clinic  and  how  they  perform  on  the  simu^itlDn, 

Finally^  the  results  of  the  medical  audit  will  be  compared 
with  the  physiuvian  results  on  the  simulatiDn.  It  is  hoped  that 
this  will  De  the  ultimate  validating  criterion.  rf  it  is  found 
that  the  physicians'  performance^  on  the  siaulation  relates  to 
their     performance     in     practice     then  some  proof     of  concurrent 


validity  can  be  estabiished. 

It  seems  a  logical  construct  to  assume  that  a  physician  who 
has  both  diagnosed  and  managed  patients  in  bDth  clinical  and 
office  sitaations  will  per£\ii.m  "better"  on  the  simulation  than 
(nedlcal  students  who  have  had  limited  experience.  It  also  seems 
logical  to  assiima  that  cardiologists  should  perforin  "better"  on 
the  si      :  .    ^ vn  then  either  of  the  above  groups, 

construct  val.taity  will  be  determined  hy  comparing  the 
perforraance  of  these  differing  groups.  Construct  validity  will 
be  tested  as  follows:  1)  the  primary  care  physicians  should 
perform  "better"  as  a  group  on  the  simulatLon  than  the  group  of 
medical  students  and  2)  the  Cardiologists  should  perform  "better'* 
as  a  group  on  the  simulation  then  either  the  physician  or  student 
jroup.  If  the  simulation  is  able  to  differentiate  the 
performdncy  of  these  differing  groups^  then  it  may  be  said  with 
some     certainty  that  the  simulation  posrvesses  cDnstruct  validity. 

It  is  assumed  that  there  iii  a  significant  difference  betwee^n 
the  groups  tested.  This  will  be  determined  by  comparing  the 
perforpance  of  the  student  and  primary  care  physi.^'ian  groups  on 
the  wri  ttf-^n  cognitive  instrument  It  is  assumed  that  the  group 
of     car  ists     is  significantly  different  than  the  above  two 

groups,  significance     is     not     founds     then     validating  the 

simulation  by  this  means  is  not  appropraite. 

The  content  validity  of  this  simulation  will  be  determinf^^ 
by   ci;.  expert   team  ol       .rdiologir;  ts , 

Scoring^gi ^ComEUter^Basea^E^^ 

One     of     the     most  crucial  and  fundamnnt;-    problorns     of  any 


evaluative  simulatLon  is  scoiring.  All  the  external  validatiiig 
instruments  that  tfere  discusaed  rely  on  a  reliable  scoring 
flaechanism,  NO  mattsr  how  well  these  instruments  are  constructed, 
and  are  psychometrica lly  sound,  if  the  scoring  ot  the  simulation 
is  not  both  reliable  and  realistic  the  validity  can  not  be 
determined.  Nor,  doess  it  matter  how  much  face  validity  the 
simulation  possessas  if  criterion  related  validity  can  not  be 
shown.  Critical  to  proving  validity  of  this  simulation  is  the 
development  of  a  sound  scoring  algorithm. 

It  is  crucial  that  the  scoring  algorithEQ  realistically 
represent  the  clinical  ramifications  of  treatment.  The  purpose 
of  SGoring  the  simulation  at  each  data  option  point  is  to  obtain 
an  overall  numeric  score  on  the  simulation  for  each  doctor  and 
also  obtain  a  series  of  subscores  on  each  sectiDn  of  the  prograra* 

This  simulation  contains  a  multitude  of  major  data  option 
point;.!  dll  of  shich  will  never  be  epcountered  in  any  one  patient* 
A  \jor  data  option  is  defined  as  a  'irBat^-^t  list  which  is 
encounted  when  the  doctor  opts  for  mea.^cal  or  uurgical 
intervention^  or,  as  a  management  list  which  usa/ly  contains  a 
list  of  options  allowing  the  doctor  to  perf^roii  a  physical^  take  a 
history,  order  laboratory  tests,  and/or  b^^^jin  medical 
intervention • 

The  scoring  will  be  accomplished  in  two  independent  phases. 
The  first  phane  will  b^  to  have  a  number  of  cardiologists  take 
the  f^i  miila tion .  Tha  in  ..vidual  physician  paths  will  be  monitor^ia 
and  placed  in  a  file,  Thene  f  ^  les  will  CDnt^l^t  of  individual 
physician  paths  and  a  cumuldtive  freguency  record  of  all  the  data 
options  that  were  choosen. 


Once     these     physicians  have  gone     thr^Migh     the  siraulation^ 
their     paths     will     be     reconstructed  on  paper     in     the     form  of 
decision     tables.     Since  there  is  no  one  optinial  path  through  the 
simulation,       the       cumulative       paths       determined       by  these 
cardiDlogis ts  cQsults  wil].  serve  as  the  standard  ••optimal"  paths. 
Once     these     atdndards  are  established/  a  weighting  scale  can  be 
instituted.     It     is     important  to  note  that  for     any     [najor  data 
'Option     there     oxist     a  multiple  nnmber  of  routes  that     could  be 
talCBn  to  reach  that  Dption  point.     In  other  words,  it  is  possible 
to     reach     a     data  option  by  one  route  and  choose     drug     'A*  for 
treatiiient     and     rGceive   Kh^'  maximum  weighted  scoret     It     is  also 
possible     to  reach  that  same  data  option  and  choosing    the  exact 
same  drug,  by  taking  a  different  and  less  desirable  route  and  not 
receive     the  maxj.mum   weighted  score.     Ihis  indicates  the  need  to 
standardize  optimal  pathways  first. 

There  are  hundreds  of  treatment  and  management  lists 
incorported  within  this  simulation.  It  would  be  almost 
impossible  ^c:  a  group  of  experts  to  weight  (score)  every  option 
within  every  data  list.  It  is  because  of  this  roallty  that  the 
decisioii  was  rnad  to  limit  and  identify  only  the  major  data 
option  points  throughout  the  simulation*  These  data  options 
constitute  critically  important  points  for  diagnosis^  treatment 
and  mangeinent*  It  is  these  major  data  options  that  will  be 
weignted . 

A  decision  table  will  be  constructed  for  each  major  data 
option  point  and  all  pD^;;^:iDle  routes  that  could  have  been  taken 
to  reach  that  particular  option  will  be  specified.  A  10  point 
walghtin  ,     seal.)  will  be  used  to  judge  the  plausibility     of  each 


dat  majora  option  within  the  simulation.  S  t<nam  ftKpPt  +  si  will 
be     efflployed  to  rate  EACH  major  data  optiL>n  on  a  '-'^  to  scale 


%  S  cr  itical  (asseatial  to  do)  to  -5  v^^ry  lfiT%:'aL  (cs,gen I::?  a  L  ^  o 
avoid)^  ,  From  the  predeterminu^d  standard  the  +5  rQLgi.^Qd  BuoLt^: 
will  be  cited  for  each  specific  option  on  the  Dptiai  il  ^  uth.  It 
vlll  be  the  job  of  this  team  to  weigh  the  other  options  for  each 
ni  '  "ior  lata  option  list.  It  is  quite  possible  and  probable  for 
any  one  option  within  a  data  option  list  tD  hav^e  differing 
weights  due   to  different  paths. 

While  the  scoring  is  consideraLly  complex^  it  is  essential 
that  it  be  done.  The  team  of  experts  will  work  independently 
rating  each  option  of  each  major  data  option  list;  the  results  of 
the  ratings  will  be  compiled,  and  the  final  ratings  for  each 
uption  will  be  determined.  If  the  re^=  ngs  are  not  consistent  on 
any  one  item,  the  highest  and  lowest  ratings  will  be  discarded; 
and  the  remaining  scores  will  be  averaged  to  determine  a  final 
weijht  for  the  option. 

I§lisbiliil^of  _.221Ettter_Based_3im 

The  last  measurement  consideration  is  that  of  assessing  the 
reliability  of  this  simulation.  The  ideal  way  of  measuring 
reliability  is  to  construct  and  develop  two  parallel  ^n^aluative 
simulations  and  loote  at  individual  performance  on  these  parallel 
forms*  Not  only  is  it  extremely  differcult  to  develop  two 
parallel  slmulatiofis  measuring  the  same  knowledge  and  skill 
levels,  hut f  this  form  of  reliability  estimation  falls  victim  to 
all  the  intervening  and  confounding  variables  of  any  parallel 
forms    situation.     This     procedure  is  also  extremely     costly  and 
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time  consufflinj,  considering  that  it  took  one  and  a  half  years  to 
develop  this  present  simulation. 

Another^  move  viable  procedure  ior  establir»hing  reliability 
is  using  a  test-retest  design,  While  this  is  certainly  possible, 
this  procedurti  is  suseptable  to  all  the  intervening  and 
confounding  variables  that  this  estimation  fresents, 

internal  rQliability  estimates  such  as  Crobach^s  Coefficient 
Alpha  or  KH20  all  make  the  fundamental  assumption  that  the 
iadividual  test  items  are  discrete  and  identifiable.  That  is,  a 
50  item  exam  has  50  dicrete ,  individual  test  items*  This  is  not 
the  case  in  an  evaluative  simulation.  There  are  no  test  items 
that  can         individually  identified, 

McGuire  and  Babbot  {19b7)  in  their  article  entitled 
"Simulation  techniques  in  the  measurement  of  problem  sDlving 
skills"  (1)  havB  looked  at  this  issue  of  reliability.  They  state 
t^at  there  ar^  four  characteristics  that  have  the  effect  of 
!  ^^.u^lng  reliability  as  it  is  commonly  estimated^  1)  the  data 
opcions  are  differentially  weighted  2)  data  options  a^^e 
interdependent  3)  there  are  di.f f erential  amrun'^^-K  of  feed  pack 
obtained  by  the  examinees  ^ihich  results  in  lissimi larity  imong 
the  students  with  respect  to  the  nature  of  the  problem  posed  by 
any  given  data  option  and  4)  an  individual  student  can  be  denied 
the  opportur;ity  of  cesponding  to  man/  of  thu  data  options  because 
of  the  particular  decisions  that  s/he  opts  to  make  (i.  e,^  may 
follow  niany  different  alternative  paths)  . 

It  Is  because  of  the.:?  four  factDrs  that  reliability 
estimation  is  extremely  diifercuit,  !icn!i.rrD  and  Babbot  tak^  the 
stance  that  reliablity  of  a   Moa^^rm  can  he  viewod  as  an  ^ittribute 


relatefl  to  it's  jeneralizabilit j  with  respect  to  different 
universeSi  Of  interest  are  the  estimates  of  the  extent  to  which 
a  particular  set  of  scores  on  the  siraulation  is  general izable  to 
many  possibl©  similar  tests*  Here  they  are  referring  to  the 
characteristic  that  Cattell  has  inclnded  in  his  concept  of 
"consistency  of  measurement''  and  which  he  specifically  refers  to 
as  ^'homogeneity"*  They  recommend  using  any  of  ti.€  three 
following  techniques  which  have  yielded  esse illy  similar 
results  from  their  own  studies;  Angoff  formula  j .  Cronbach's 
Coefficient  Alpha,  and  principal  component  ana^  . r  ;  ,  ^11  will 
estimate  the  consistency  of  measarement  or  general izability  of 
results  across  tests* 

While  it  is  possible  to  obtain  a  reliability  estimate 
according  to  McSuire  and  Babbot,  the  four  characteristics 
inherent  in  a  simulation  that  reduce  reliability  have  yet  to  be 
resolved,  it  seems  possible  that  a  coefficient  can  be  derived 
which  is  sensitive  to  the  multitude  of  confounding  variables  that 
exist  in  this  teehrigue*  what  is  needed  is  more  theoritical 
research  in  this  area  by  those  familiar  with  this  technigue  and 
the  areas  of  reliability  estimationi 

Summari^ 

In  sumniation^  evaluative  computer  based  siDnulations  are  both 
exciting  and  challanging  ^.^urement  instruQients  for  any 
eduaction  environment,  Thji.s  mode  of  evaluation  offers  an 
insiovative  means  Df  assessing  a  learners*  conpr ehension  r 
Knowledge^  and  skill  levels  within  any  subject  area.  While  there 
ii     pany     unr^aaolved  meusurement  issues^  the  foundation  is  laid 


for  researchers  to  e.rplore  these  vital  area  of  interest. 
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ABSTRACT 

The  paper  presents  the  resiilts  of  two  formative  evaluation  studies 
of  second  language  programs  in  Nova  Scotia*     Both  studies  included 
procedures  to  detarmina  the  effectiveness  of  the  series  in  achia%?in9  their 
objectives  as  well  as  procedures  which  could  yield  suggestions  for  im^ 
provement  in  the  programs. 

second  aspect  of  the  paper  concerns  the  imp covement  of  f amative 
evaluation  procedures  for  television  and  other  mediated  prograins*  The 
suggested  changes  are  divided  inwO  factors  which  can  be  examined 
indeper.dently ,  factors  which  must  be  examined  in  con  text  and  factors 
which  must  be  examined  in  a  f  eld  tryout* 
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The  educational  media  liavs  a  definite  role  to  play  in  Helping  children 
and  adult.^  to  learn  other  languages*    Radio ^  audio- tapes ,  films  and  slide 
sound  pacKages  and  televiaiDn  have  all  been  used  to  achieve  y-arious  learning 
objectives*    Many  of  the  earl^  language  niedia  presentations  consisted  of 
stand  up  teaching  using  blacteoarde  and  perhaps  a  few  visuals  but  little 
else.     However iT  with  the  rtiatuJcation  of  educational  televisiori^  more  sophia* 
ticated  presentations  are  hairig  designed  to  teach  languages. 

Many  ^gtincies  have  prodycedi  series  for  use  in  second  language  teaching. 
The  Ontario  Educational  Conrnmioations  Authority  in  Toronto^  has  produced 
several  satias  for  ust  in  teaching  Firench  while  the  Carrascolendas  series 
has  heen  ased  to  teaoh  Spamr^h  In  the  United  States,    Nova  Scotia  Schoal 
Television  also  produced  scivecaL  seri#a  to  suppleinent  teaching  o£  Frarich 
as  a  secorid  language* 

Most  of  the  racent  tele^Lslon  stales  intended  for  second  language 
learners  have  been  subjected  tq  forma tive  evaluation  or  at  least  surrsnative 
evaluation  procedures.    These  sfcudiei  have  deait  with  various  aapecta  of 
the  prdductlon  and  utiliMtiOT  of  second  language  television  materials* 
(See  Laosa,  1975|  ^miims.  Van  Wart  and  Stanford,  1973i  Fleitiing,  1972) 

This  paper  deals  with  tte  Improvement  of  formative  evaluation  procedures 
for  second  language  televlsioa  materials*  The  recommandid  procedures  are 
based  on  a  review  of  the  literature  and  on  two  studies  of  second  language 
teaching  mttarials  produced  toy  ^ova  Scotia  School  Television  (Lewis  and 
Fisher^  1977;  and  Lewis,  1979)  .  Although  the  paper  deals  with  procedures 
for  television  programs^  ^e  coMients  could  be  applied  to  other  media  and 
other  progtams. 


The  Nova  Scotia  Studies 

The  recoimnendatlons  on  improving  procedures  are  based  on  two  studies  of 
second  language  talevision  programs,     To  provide  the  reader  with  this 
background,  these  studies  are  briefly  suinmarized  below. 

Vive  La  Compagnie 

Vive  La  Compagnie  was  a  second  language  French  seriai  d^,£igned  for 
grade  eight  students.     In  each  program,  three  actors  (and  other  supporting 
actors)  participated  in  real-life  sKperiencea  which  might  be  facec^  by 
students*     The  programs  had  a  controlled  vocabulary  and  relied  on  humor 
and  visuals  to  convey  meaning* 

The  Evaluation 

The  evaluation  was  commissioned  by  the  Nova  Scotia  Department  of 
Education  to  determine  the  effectiveness  of  the  progr^  aeries  achieving 
iti  objeotives,  which  dealt  with  vocabulary,  acquisition,  enjoyment  and 
understanding  of  the  plots.     In  addition  to  detennining  effectiveness,  the 
study  had  to  provide  specific  information  on  reconmended  productlan  changes i 
diitribution  patterns,  usage  guidelines  and  supplTOent^ry  materials. 

Spacific  Questions 

1.  Would  the  students  watch  the  programs i  l*e.  would  their  eyes  be  on  the  screen? 

2.  Would  the  students  understand  tha  %^ords  to  be  used  in  each  program? 

3.  After  watching  the  progrms,  would  students  understand  words  ^hich  were 
not  in  their  repertoire  before  the  viewing? 

4.  Would  students  ^derstand  the  plots? 
5*    Would  students  enjoy  the  prograJrts? 
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§•        Would  the  number  o£  programs  watched  result  In  differences  in  v^qcabulary 
acquisitioni  uander standing  and  enjoyriient? 

7.  How  would  studarits  react:  to  selected  v^ariables  such  as  the  plot, 
the  character  and  p^odutction  techniquea? 

8.  Would  the  students'  attitudes  to  learning  French  and  towards  French 
people  be  alteied  by  eHposure  to  the  prograti? 

9.  How  would  the  teachers  feel  about  the  program  with  regard  to  its 
levels  its  achievement  of  objectives ,  etc. 

Data  Sourceg 

The  avaluatiQa  study  used  three  basic  sources  of  data:     aKpert  opinioa, 
student  opinion  and  test  rasults,  and  teacher  opinion.     The  eKpert  opinion  oon- 
sistad  of  a  contaHtual  evaluation  in  which  the  programs  were  scrutinized  to 
determine  the  produotion  variables  which  could  be  improved.    The  actions  of 
the  characters,  the  pronunciation  and  the  special  effects  all  came  under 
closa  sGrutlny  for  iuitability.     In  addition,  the  relationship  of  the 
produotion  fomat,  %ta,,  to  existing  research  recommendations  for  children's 
programs  was  investigated.    The  researcher  determined  whether  the 
reoomnendations  of  experts  in  children's  progranuning  were  being  considered 
in  the  Series. 

Teacher  attitude  was  obtained  using  a  Llkert  type  scale  similar  to  the 
students'  scale,    la  addition,  each  participating  teacher  was  interviewed  to 
determine  his/her  retGtion  to  every  a^ect  of  the  program  which  could,  if 
necessary,  be  changed  or  altered* 

Design,  Subjects,  progedwi 

Two  hundx^ed  aiid  eighth-eight  grade  eight  students  from  12  classes 
participated  in  the  study.    These  atudents  came  from  fow  different  schools. 


Classes  were  chosen  at  random  from  all  the  grade  eight  Pjranch  classes  in  one 
Nova  Scotia  county.     To  dateEmins  the  effectB  of  watching  mora  programs, 
SQine  classes  watched  eight  programs,  some  watched  four  and  aotne  watched  one 
Pffograjn;  other  classes  did  not  watch  any  programs  but  participated  in  the 
study  so  that  valid  comparisons  could  be  made.    All  claaaaa  cornpletad  a 
basaline  teat  battary  bafora  the  study  began  while  one-half  of  the  sarnpla 
cowpleted  a  pre-teat.     Classes  viewing  the  program  watchad  it  on  videotape 
in  their  olassrooms  during  the  regular  Prench  class.    Discussion  of  a  program 
Eollowed  its  showing.    After  watching  the  appropriate  nurnber  o£  programs,  all 
the  clftssas  were  tasted  on  the  same  progrtim  using  the  techniques  mentioned 
above.     To  obtain  the  most  representative  aampla  of  taaclior  opinion 
regarding  the  program,  any  tsachar  using  the  program  outaide  the  study  as 
well  as  several  other  groups  of  teachers  complatad  the  teacher  questionnaire 
on  the  series. 

pre-Tgeatment  Tests  for  Diffegennea 

En  this  study,  classes  had  to  be  randomized,  not  atudents.     As  a 
result,  differBnoes  betwaan  cLaases  had  to  be  measured  hefora  the  treaOiieiit 
was  pre:9ented.     Two  tests  ware  administered  to  the  sample  prior  to  tha  study. 

The  loi  on  Parle  Test,  an  audio-piettorial  vocabulary  teat,  disco'/ered 
significant  differances  between  groups.    In  every  case,  means  for  the  TV 
group  appeared  to  be  higher  man  means  for  the  non-TV  group. 

Significant  differances  betwaan  groups  were  also  Sound  in  responses 
t»  the  questionnaire  on  attitude  to  learning  French.     This  teat  comprised  two 
suiJ-tests,  Dasire  to  Learn  Prench  and  Motivational  Intensity. 


Result s* 
ftttentiQ'n 

The  progran\s  rnaintained  the  attention  of  studenti  v^fatching  about 
80%  of  thm  time.        However,  certain  sections  of  pffograns  did  setsm  to 
encourage  students  to  stop  watohing  programs.     In  addition^  it  appeared 
that  students  ^ho  did  not  understand  the  action  (deteroiined  hf  seif-report) 
stopped  watching  to  catch  up  on  their  sleep,  homeworlc,  etc.  Sections 
of  programs  wliich  were  accompanied  by  lapses  in  attention  were  noted  for 
further  analysis* 

yggabulary  Xievel  Before  Viewing 

the  144  students  who  completed  tlie  pre-test  Knew  14  of  the  20  words 
on  the  test.     This  data  indicated  that  students  could  understand  the  major 
wrds  and  sentence  eKprassion  used  in  the  program.    This  finding  showed 
that  the  assessments  by  the  teachers  and  of  the  producers  regarding  vocabulary 
level  wera  la^sonably  accurate* 

Vooabulafy  Laval  After  the  Program 

OSia  difference  between  pre- test  and  post-test  TOcabulary  scores 
in^a^cated  that  students  did  i^^ot  learn  new  words  or  phrases  as  a  result 
of  the  program.    Neither  the  pre-teit  group,  nor  the  non  pra-test  group 
who  watahed  tte  television  showed  a  significant  difference  on  the  post-test 
from  the  rpn--tslevision  groups.     However,  it  should  be  pointed  out  that 
the  program  was  not  designed  to  teach  vocahula^* 

*  This  section  contains  only  summary  information  since  the  paper  is  concerned 
mainly  ^ith  the  improvement  of  procedures.    Complete  results  are  presented 
in  Lewis  and  Fisher  (1977). 
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The  data  did  indicate  that  a  student  who  had  been  erased  to  a  pre- test 
would  score  significantly  higher  than  a  student  who  was  not  e^^osad  to 
the  pre--tegt.     As  a  result,  one  of  the  recominandations  suggested  that  as 
an  instructional  device ^   students    be  exposed  to  the  vocabulary  of  the 
program  before  it  was  viewed* 

Comprehension  of  Plots 

Students  who  watched  more  prograins  understtaod  more  of  the  action  than 
students  who  watched  fewer  prograins.     It  is  obvious  that  comprehension 
improved  as  the  students  became  accustomed  to  the  accent  and  pronunciation 
of  the  characters  in  the  Series,     a?hese  findings  clearly  demonstrated  that 
viewing  the  whole  series  would  be  more  beneficial  than  ttle  viewing  of 
just  one  program^  or  even  just  a  part  of  the  series . 

Opinion  of  the  program 

No  significant  differences  were  recorded  between  students  who  watched 
one,  four,  or  eight  programs.     In  ranking  the  programs,  students  chose 
programs  which  they  said  were  easy  to  imderstand,  funny,  presented 
pictures  to  explain  the  words  and  had  a  gool  plot.     More  infonnation  on 
these  factors  was  sought  and  obtained  in  the  group  and  personal  interviews 
after  each  viewing  and  at  the  end  of  the  study.     The  findings  of  the 
interviews  and  the  suimnary  of  the  questionnaires  provided  the  basis  for 
many  of  the  production  changes  which  ware  suggested* 

Qpinions  of  Teachers 

Most  taachsrs  indicated  that  students  would  like  the  program  but 
that  a  donsidarable  amount  of  preparatory  and  follow-up  work  would  be 
needed*    ms^  teacheri  fait  that  the  students  would  not  understand  the 
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vocabulary  presanted--an  opinion  which  was  completely  contradicted  by  the 
actual  perfomance  of  the  students,    h  strong  feeling  of  the  teachers  was 
ttet  the  pronunciation  of  one  of  the  characters  would  inhibit  learning  the 
correct  pronunciation*    As  a  result,  a  change  in  characters  or  an  improvMient 
in  pronunciation  were  suggested  reconmendations * 

Attitude 

The  viewing  of  the  program  made  no  difference  in  the  attitude  scores 
of  studanta.    Those  who  watched  did  not  attain  higher  attitude  scores* 


Problems  with  the  Study 
EKperimenter  Effect 

In  this  study,  the  es^erimenter  supervised  the  viewing  of  all  progrmtis 
and  administared  all  teats.    This  procedure  couW  hardly  be  called  a 
natural  situation  for  the  classes  involved.    As  a  result,  the  generaliMbility 
of  the  data  to  a  teacher  administered  situation  could  be  called  into  question* 

Vocabulary  Testing 

The  same  vocabulary  test  was  used  as  the  pre-  and  the  post-test. 
The  data  indicated  that  the  administration  of  the  pre-test  affected  the 
post-test  acores,  i.e.,  students  who  took  the  pre-^test  iocred  significantly 
higher  than  students  who  did  not  complete  the  pre-test. 


Attitude 

Attitude  is  difficulty  to  measure.     However,  with  the  efforts  of 
Gardner  and  Smythe  (1975)  attitude  towards  learning  French  can  be  measurad. 
However,  Gardner,  Ginsberg  and  Smythe  (1976)  noted  that  attitudes  of 
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poat-tests  seem  to  be  lowe^e  than  attitudes  of  pre-tests  where  there  is  no 
intervention  (such  as  a  tsaevislon  program) .     They  attribute  this  finding 
to  the  fact  that  subjects  tested  are  alerted  to  attitude  which  they  did 
not  have  to  eKpress  before* 

Therefore,  the  measurement  procedures  used  in  this  study  may  not  be 
as  valid  as  others  which  could  be  designed*     The  deletion  of  a  pre- test  of 
attitude  with  only  post-^test  being  administered  may  be  a  more  desitfabla 
procsdure.    Alternatively,  two  equivalent  forms  of  one  instrument  or  two 
highly  correlated  instruments  could  be  used. 


Meaaurement  of  Attention 

The  attention  measures  were  subject  to  a  great  deal  of  error  since 
it  was  impossible  to  be  certain  whether  a  student  was  attending  the  program 
or  juat  had  his  head  turned  in  the  direction  of  the  television  receiver. 

ALLOWS-Y 

T?he  Series  consisted  of  ten  fifteen  minute  programs.    Each  program 
contained  a  nmber  of  short  segments  which  offered  a  range  of  topi£=s,  The 
range  of  segments  included  appearances  by  a  teanaged  host  and  hostess, 
singers,  a  band,  gaine  shows,  activity  progrras  and  on-^site  visits  to 
locations  like  Cheticamp,  a  fishing  village  in  Cape  Breton, 


Supplementary  Materials 

Supplementary  materials  included  a  teacher's  guide  and  a  set  of 
flashcards  depicting  key  vocabulary  and  concepts  used  in  t±e  programs. 

The  teacher's  guida  included  a  list  of  the  program's  objectivas, 
information  on  the  segments  presented  in  each  program,  a  description  of  the 


skit  in  each  progran  and  selected  vocabulary  and  idiomatic  eKpression* 

The  guide,  howeveir/  did  not  include  any  suggestions  or  airections  for  using 

tha  program. 

Sixty- Sevan  flashcards  depicting  vocabulary  and  concepts  were  avallabl 
free  of  charge  to  teachers  requesting  them.    These  flashcairas  were  printed 
in  black  and  white  on  heavy  cardboard. 

Field  Testing  by  Department  Personnel 

Three  of  the  programs  were  viewed  by  students  in  20  schools  during 
visits  by  a  staff    mei^er  from  the  Education  Resource  Servioes,    After  the 
students  viewed  the  prograni^  t^ey  were  asked  to  coirunant  on  the  programs  on 
an  open  ended  questionnaire.     Some  of  these  corwnents  are  presented  in  a  pap 
by  Johnston-Doyle  and  Lewis  (197S) , 

PurpQge  of  the  Evaluation 

The  purpose  of  the  evaluation  was  to  determine  the  effeotiveness  of 
the  series  in  aohieving  its  objectives  and  to  make  conareta  reconmendations 
for  ehanges  in  the  production  and  utilizations  of  the  programs- 

Specific^  Quegtigjgg 

1.  Would  viewing  the  program  have  an  effect  on  students'  ability  to 
Gomprehend  words  and  sentences? 

2.  Would  the  program  help  students  recognize  correct  constructions? 

3.  Would  the  students  understand  the  content  presented  in  the  segments? 

4.  Would  the  students  identify  with  the  teen-aged  host  and  hostess  in  the 
program? 

5.  Would  the  students  like  the  musical  segments  presented  in  the  program? 


6*      Would  attitude  to  learning  French  be  affected  by  visiting  the  progtam? 

7.      Wouid  attitude  to  the  French  course  be  aff acted  by  vim^inq  the  program? 

8-      Would  the  addition  of  a  teacher's  prepa^atoEy  and  follow-up  work 

affect  scores  on  comprehension,  ability  to  yacognize  constructions^ 
attitude  to  learning  Franch  and  attitude  to  the  French  class? 

9*      What  chMges  in  the  program  could  be  reconunended  as  a  result  of  the 
testing  and  the  opinion  of  teacher  and  students. 

The  study  was  divided  into  four  sub-^Btudies  in  order  to  answer  the 
variety  o£  quastiona  posed  by  the  clients*    The  first  study  concerned  itself 
with  the  effect  of  viewing  the  progrms  without  any  teacher  input*  The 
second  study  monitored  the  effects  of  the  television  program  and  teacher 
input*    The  t^ird  study  dealt  with  the  difference  between  the  two  groups 
in  the  first  two  studies.    The  final  study  iiarveyed   attitudes  towards  the 
programs* 


Sample 

The  studies  used  populations  of  gracle  eight  students  in  Nova  Scotia, 
In  studies  one  and  two,  an  es^eriniental  groijp/ control  group  (Viawing/uQn- 
viewing)  design  was  used  with  the  same  teaahers  teaching  both  a  viewing 
group  and  a  non-viewing  group.     In  study  one,  tAe  experimental  group 
(78  students)  watched  the  prograJn  without  tsachei'  input  while  thm  control 
group  (92  students)  continued  with  the  specified  curriculum*     In  study  two* 
the  52  viewing  students  watched  tha  program  and  participated  in  teacher 
praparatcry  and  follow-up  activity,  whil©  the  20  control  group  students 
continued  with  the  curriculum.     In  study  three,  the  52  students  who  viewed 
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with  the  teacher  were  eompared  to  the  78  students  who  did  not  have  teacher 
input.  Study  four,  which  sought  to  measure  attitudes  and  opinions,  includei 
177  students  who  had  viewed  the  programs* 


All  students  completed  a  baseline  battery  which  included  a  test  of  French 
comprehension  and  vQcabulary,  an  attitude  test  and  a  basic  information 
questionnaire.     Then,  viewing  groups  watched  eight  progrOTS  while  non-viewing 
groups  proceeded  with  their  regular  work.    After  viewing  the  programs,  all 
students  completed  the  post-test  battery.    Viewing  students  completed 
a  vocabulary  test,  a  constructions  test,  a  sentence  comprehension  test, 
a  recall  of  skit  test  ,  three  attitude  tests  and  the  tests  on  the  opinion 
of  each  segment  of  the  program.    Non-viewing  students  conipleted  only  the 
vocahula^^  sentence  cpmprehension,  constructions  and  three  attitude  tests. 


Analysis 

The  analysis  of  covariance  procedure  developed  by  the  Division  of 
Educational  Research  Services  at  the  University  of  AllDerta  was  used  to 
process  the  data.    The  analysis  is  computad  on  the  basis  of  a  pooled 
regression  equation,  homogeneity  of  regression  is  assumed.     This  procedure 
was  used  for  studies  1,  2,  and  3. 


study  1 

The  purpose  of  study  1  was  to  deterTnine  v^hether  eKpoaure  to  the  Series 
woald  result  in  an  increase  in  poit-tesG  scores  on  ability  and  attitiide. 

B a s el in g  Mgas ur e s 

Tha  two  groups  (nQn^viewing  and  viewing)  were  not  significantly 
different  on  the  baeeline  measures*     (See  table  1).    On  the  lei  on 
Parle  Teat,  a  test  of  Ftenah  ability^  the  viev?lng  group  scored  25.31 
while  the  oontrol  group  scored  23.65,    Both  groups  scored  between  60 
and  70  percent  on  the  test.     On  the  attitude  test  the  viewing  group  scored 
60.24  while  the  non-vtawing  group  scored  56,53. 

Pos t^ te  St _  Measureg 

The  scores  of  the  two  groups  were  significantly  different  only  on  the 
vocabulary  test,  after  adjustatent  for  the  effect  of  the  covariatea 
measured  in  the  baseline  teat*     (See  table  1) .    The  groups  ware  not 
significantly  different  on  the  sentence  comprehension  test,  the  construction 
test  and  the  three  attitude  measures. 


for  thi  vi.iwj.ng     non-viatfkg  g^oiips  m  ill  tagts 


Uriiijysted  Meini 


jajus  tid  Mains 


Teit 


lei  Oil  Brio  tml 


25. 3L       ^3.65        2.62  1/163 


ie  Test  L 


iO,24       §5.53  3.01 


Seiitiiiee  CoinprilieriilQn  Test     ^^^^  l^^g 


Vocabulary  Teic 
(fiiK*16) 

tonsMiictions  Test 
Attitude  Teit  2 


fttti tilde  Test  3 

mtitm  Test  4 


5,06 


3,81*  l/jS8  U?  ^.33  .99 

8.2i*  1/J68  5.59  4.-16  5.20* 

,11  l/j68  5JS  2.80 

€.ai*  l/JeS  41.65  3.21 

L31  1/168  90. 4&  96-54  1.85 


1.4fi  4,05' 


study  2 

The  purpose  at  atuay  2  was  to  deterinine  whether  es^osure  to  the  Seriss 
would  result  in  an  increase  in  lost-test  scores  on  ability  and  attitude. 
However,'  in  this  study,  the  tefl.chac  actively  workad  with  the  class  before 
and  after  the  viewing,  relnfoxeiag  the  content. 

Baseline  Batterias 

The  viawing  group  scored  signlf iaantly  higher  than  tha  non-viewing 
group  on  the  Ici  on  Parle  teit.      (See  table  2) .     The  mean  of  tha  viewing 
group  (31.85)  was  four  pointi  higher  than  that  of  the  non-viewing  group 
(27.85). 

post-tests  . 

The  scores  of  the  two  groups  were  signifioantly  difflersnt  on  two  testa, 
the  vocabulary  test  and  the  sentonce  coraprahension  test  (See  table  2) .  The 
scores  on  the  other  tests  were  not  significantly  different,  affter  adjusting 
for  the  covariates. 
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Tabla  2 


Unadjusted,  idjuitei  inians  and  f .  Ratios 
for  the  Viewing  ind  non-viawing  groups  on  all  tists 


unadjmtsd  Means  ^djustid  Means 


Test 


Viswing  Non-Viewing  P.  Ratio  Viewing  Non-Viewing  Ratio 


lei  on  Parle  Tsst  j^^g^      ^^^35      14J3*  ij% 


Attitude  Teat  1  gj^^^      ^^^qq        jg  y^O 
(Mx-90) 

Sintenci  Coniprehinilon  Test     ^^q^  ^^^g       3^j^*    y^p  5.37  3.88  24.85* 

Vflcabulary  Tiit  4jg  i^jg*  i/%  11,49  4. 83  61.62* 
(HiK-16) 

Constructioni  Teit  ^^^^  gjg  4j3*  y^o  7.93  1.W  .76 
(!4ax-14) 

MtitudiTieti  ^^^29      45,10        .00     l/?0  45.69  44.07  .72 

Attitudi  Tilt  4  g^^^p  ggjQ  ,og  1/70  88,96  89.93  .20 
(llaxnSS) 

Attituds  Tilt  4  ^Q3jO  ,14  1/70  104,63  103.  26  .19 
{ta-180) 


*  p>.05 


study  3 

The  purpose  of  study  3  was  to  detarinine  tha  effect  of  a  teacher 
on  ths  post-test  icores*     In  the  study ^  ona  group  {T^)  watched  the  program 
without  any  follow-up  or  preparatory  activity  while  the  other  group  (TVT) 
partialpated  in  activities  in  addition  to  viewing  the  program. 

Ba^aline  Test 

The  W  and  Teacher  (TOT)  group  scored  significantly  higher  on  the  Ici 
Vocabulary  Teat  than  the  TV  only  (TV)  groi^*     (See  table  3).     However,  tJia 
mo  groups  were  not  significantly  different  on  the  attitude  battery*  At 
31,85^  the  mean  for  the  TVT  group  was  higher  than  the  meafi  for  the  TV  group. 

Vocabulary  Test 

The  TV^r  group  was  significantly  different  from  the  W  group,  (See 
table  3).    The  adjuated  icoras  ware  6.51  for  the  TV  gro\jp  and  7,46  for  the 
fVT  group* 

The  efforts  of  the  taachar  obviously  resulted  in  Che  increased  ^Qca.bula2Y 
level  of  the  group  who  had  the  teacher  adding  her  input  to  the  eKperiena^i* 

Sentence  Comprehenaion  Test 

The  adjusted  scores  for  the  two  groups  were  not  significantly  different^ 
(sea  table  3).  The  TVT  g«up  icored  at  50%  (6,0)  before  adjustment  while  the 
TV  group  scored  at  30, 5%  (4*74), 

The  efforts  of  the  teacher  did  not  appear  to  significan^y  affect  the 
results  of  the  sentence  Gomprehensicn  test. 
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Construe tiorii  test 

The  tw  groups  were  significantly  different  on  the  constructions  test. 
(Sae  table  3).    When  the  scores  were  adjusted,  the  WT  group  scored  9*99 
while  the  W  groi^  soores  7.16*    Once  again  the  teacher  preparatory  and 
following  activitiss  appeared  to  have  a  positive  effect  on  the  ability  of  th 
group  to  recognise  correct  conatruotions. 

Attitude  Tests 

On  all  three  attitude  tests,  the  groups  did  not  differ  significantly, 
(See  table  3) *  indicating  that  the  intervention  of  4  teacher  along  with  the 
telavision  Be:kim3  did  not  significantly  affect  students^  opinione* 


Table  3 


Adjusted  and  unidjuited  means  for 
TV  only  and  TV  and  Teachar  Groups  on  all  variables 


Unadjuited  mm  ^^i«sted  Means 


TV  Only  F.  Ratio     df     TV  Only     f  .f^        Ratio  df 

test  ^gjgj^gj  Teacher 


Ici  on  Parle  ^ist  ^SJl       31.85       31.43*  1/128 


Attitude  Tiat  1 


60.24       63.11        in  1/128 


Stntance  Corapreheniion  Test  ^  „  ^^^^g  ^^^j  gjg  .57  1/12S 
(Ma3fcl2) 

Vocabulary  Test  ^  ^  29.19*  1/UB  6.51  ?.46  6,86*  1/126 
(Ma)cml6) 

Constructions  fast  ^^^^        ^^^^^^       ^Q_3g,     y^^e  1.16        9.99  2^.26* 


Attitude  Test  2  j_28      1/128  41.19  44.23         .62  1/126 

Attitudi  Tast  3  y^5g  gq^^q  09^7  .00  1/126 
(MaxJ.55) 

Attltudi  Tist  4  ^Q^^^^  1/128  103,44  105.65  1.13  1/126 
(MaxiSO) 


study  4 

The  purpose  of  study  four  was  to  obtain  studetit  opinions  of  the  various 
production  and  content  a^ecte  of  each  of  the  segments.     The  overall  findings 
seem  to  indicate  that  for  the  most  part,  with  the  exception  of  the  Cheticanp 
sagments,  the  students  wete  able  to  follow  the  action  and  understand  what 
was  going  on.    Most  of  ths  segments  were  coniidered  quite  enjoyable 
by  the  students.     They  reacted  positively  to  the  teen-aged  host  and  hoitess 
but  said  that  some  of  the  games  on  the  game  gegments  were  probably  more 
suited  to  yQunger  children.    The  segment  which  received  the  highest  overall 
rating  was  thm  skit  sa^ents,  niainly  beeausa  of  the  humo^  and  the  quick 
action* 

The  analysis  of  the  recall  of  segment  answers    indicated  that  most 
students  caught  the  general  drift  of  the  action  but  missed  important  details, 

A  variety  of  e^^erim^ntal  designi  and  procedwes  are  needed  in  order 
to  conduct  a  formative  evaluation  of  a  second  language  or  indsed  o£  any 
television  project* 

One  of  the  nost  inportant  factors  is  the  e^ert  evaluation  in  which 
people  who  qualified  for  tha  test  review  tim  ptogtm  independent  at  students 
to  dete«iine  the  answers  to  questions  such  as  the  suitability  of  the 
vocabulary,  the  aecenta,  the  correspondence  with  the  ct^riculum,  the  plot 
level/  etc. 

Another  procedure  Involves  the  use  of  a  few  students  to  determine 
aBpiriGally  the  vocabulary  level  of  the  program  in  comparison  to  the  level 
of  the  target  audience,  the  attention  of  students  in  small  groups  and  other 
such  factors. 


The  field  ivaluation  involves  the  actual  in-class  testing  of  the 
program,  with  teacher  input  and  withouti  to  determine  the  program' s  ability 
to  achieve  its  intended  objectives •    This  type  of  procecurt  was  cjuite  widely 
used  in  most  evaluations. 

Most  evaluations  which  included  field  testing  use  e^cperimental  groups 
control  group  designs  with  or  without  some  provision  for  randomization  or 
statistical  control  of  pre^treatrnent  differences. 

In  evaluating  second  language  materials*  some  or  all  of  the  factors 
mentioned  below  niay  have  to  be  included*    The  factors  are  divided  into  three 
categoriesi     factors  which  can  be  ejcamined  by  viewing  the  program* 
factors  which  need  viewers  interacting  with  the  program. 

Factors  which  can  be  exOTlned  independently 
1,    Vocabulary  level  of  the  program 

The  vocabulaa^  level  of  the  program  must  be  within  the  range  of  the 
students  in  order  that  they  can  benefit  from  the  program.    There  are  several 
ways  of  determining  ideal  vocabulary  level*    One  way  TOUld  be  to  compare  the 
list  of  words  used  in  the  program  with  the  list  of  words  uaad  in  the  instruction 
materials  which  are  being  used  in  the  qurriculum*    Another  means  might  ba  to 
pre-test  the  students  on  the  vocabulary  which  is  used  in  the  program*  The 
pre-test  can  take  the  form  which  is  used  in  curriciilum  materials  which 
usually  consista  of  a  series  of  pictures  in  which  the  student  circles  thm  correct 
alternative  which  is  mentioned  on  an  audio-tape.    This  tj^e  of  tsst  was  used 
in  both  the  "Viva  La  Compagnie"  and  the  "Allons-Y"  evaluations,    A  variation 
was  used  in  thm  ••Allons-Y"  evaluation  in  which  the  student  numbered  one  of  twenty 
pictiitfes  with  the  number  of  the  phrase  or  word,    Howevtr,  this  test  was  a  bit 
more  difficult  than  the  traditional  test  in  which  the  student  circled  one  of 
four  alternatives, 
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2.  Th§  types  of  constructions  used 

The  formats  of  constructions  will  affect  comprehension  of  atudants. 
The  formats  of  construction  used  in  the  program  must  match  the  forniats 
which  have  been  used  by  the  studBnts  in  their  curriculum  or  which  have  been 
reinforced  In  the  teaching.    Once  again,  the  types  of  conatructions  which  the 
students  understand  can  be  tested  or  can  be  deduced  from  the  program  niaterials. 

3,  Accents 

The  accent  of  the  actors  must  match  those  which  the  studanta  are  used 
to  hearing.    However,  there  is  some  discussion  on  this  fact  with  one  camp 
believing  that  the  studant  must  get  used  to  hearing  a  variety  of  accents 
while  the  opposite  view  holds  that  the  student  must  hear  words  with  the 
same  accent.    This  is  proven  to  be  an  especially  vexing  problem  in  Canada 
where  the  French  spoken  in  the  various  regions  differs  in  terms  of  accent 
and  also  differs  from  french  spoken  in  France.    Similarly,  the  types  of 
inflections  placed  on  words  must  be  similar  to  those  heard  by  the  students. 
One  of  the  ways  this  task  can  be  done  is  to  ask  expert  personnel  to  judge 
the  accents  and  inflections  or  to  pre-tist  on  a  small  number  of  students. 
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4.    laiaq^e  presented  by  the  actors 

In  any  materials  which  have  as  a  goal  the  in^rovement  of  relationships 
between  language  groups,  the  image  presented  by  the  actors  in  the  second 
language  program  must  be  very  carefully  inonitored.    Everts  in  the  area 
must  ba  %Bked  to  evaluate  the  imge  portrayed  by  the  actors.    If  the  actors 
or  aotresses  exhibit  a  ntgativa  image  to  the  stydents,  this  may  well  inhibit 
learning. 
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5 .  Contribution  of  production  effects 

The  sets,  and  production  techniques  should  be  related  to  the  objectives 
of  the  program.     In  addition,  these  factors  should  be  similar  to  the  kinds 
of  television  to  which  the  students  are  accustomed,     Nakeshift  sets  are 
detected  very  quickly  by  students  and  cannot  be  used  if  the  purpose  is  to 
present  professional  television*     In  addition,  the  sets  should  not  in  them- 
selves distract  from  the  objectives. 

6,  Correspondence  of  audio  and  visual  message 

The  presentation  of  the  new  TOcabulary  words  can  and  should  be  accoit^janied 
by  their  equivalent  in  a  real  referent.    When  the  French  or  Spanish  word  for 
cup  is  used,  the  cup  should  be  the  most  prominent  object  on  the  screen*  Another 
method  would  be  to  use  a  visual  cue,  with  perhaps  QM  of  the  actors  pointing 
to  the  ci^,  to  indicate  precisely  which  object  is  being  referred  to  in  the 
audito:^  message*    In  one  of  the  programs,  toe  weak  correipondence  between 
audio  and  visual  stimuli  meant  that  students  did  not  dltarly  understand  which 
objects  an  auditory  message  was  referring  to. 


7.    speed  of  tAe  actors*  speech 

Thm  actors  must  speak  at  a  pace  which  is  correct  for  the  students.  If 
the  pace  is  too  quick  the  students  will  not  understand  the  speech.  However, 
if  the  speech  is  too  slow^  It  se^a  unnati^al  and  will  distract  the  students 
in  other  ways. 


8*    Level  of  content 

If  a  story  is  being  used,  the  level  of  the  plot  must  match  the  interest 
level  of  the  students.    If,  as  in  "Allona--Y*%  a  segment  approach  la  used,  the 
segments  th^selves  muat  also  match  the  interest  of  the  students. 

ERIC  °  -  ^ 


9 .  Complexity  of  plot 

The  plot  must  be  oomplesc  enough  to  hold  the  students'   interest  and 
yet  siinpla  anough  so  that  the  students  will  .understand  it.    Teachers  and 
other  proeassionals  can  judga  the  complexity  o£  plot  relative  to  a  group, 
however,  a  better  means  might  be  to  use  small  groups  of  studants  and  pilot 
studies  to  detarmine  the  suitability  off  the  plot  to  the  intendad  audience. 

10.  Analysis  of  the  grograin  into  infotfnation  units 

Using  a  procedure  outlined  by  priadlarider  (1974)  the  prograin  should  be 
broken  into  infomatlon  units  go  that  these  units  can  be  tested  in  comprehension 
tests  or  tracabulary  teats  later  on.    Analysis  of  infomiation  units  woiild 
probably  take  place  by  teachers  familiar  with  the  pEograiii  or  by  resiarchers 
who  are  conducting  the  evaluation. 

Factors  which  must  be  exainined  in  context 
1.    CQgrespondetice  _wi  th  the  eurr  iculijn 

The  television  program  which  is  used  should  correspona  in  some  way  to 
the  curriculum  which  is  being  used  in  Uia  fletools.     If  the  vocabulary  con- 
structions or  structure    of  the  program  is  contrary  to  that  of  the  instructional 
programming  used  in  the  schools,  a  difffioulty  will  anerge.    Once  again,  this 
evaluation  could  be  conducted  by  eKpert  researchers  or  by  teachers  who  are 
familiar  with  both  the  program  and  the  instructional  materials  being  used  in 
the  school.    A  subsidiary  point  is  that  the  program  should  help  fulfill 
some  of  the  curriculiMi  objectives  in  the  curriculum,    it  is  important  that 
the  objectives  of  the  program  be  soniewhat  tha  sana  as  the  objectives  of  the 
overall  curriculOTi. 


2.    How  the  program  can  be  used 

In  many  cases*   the  schedule  of  the  junior  high  and  high  schools  would 
make  it  impossible  for  the  prograin  to  be  used  on  an  off-the--air  basis 
unless  cable  scheduling  can  be  arranged,    Ttie  accessibility  of  the  program  to 
the  target  audience,   then,  must  be  clearly  identified.     In  some  of  the  Nova 
Scotia  evaluations,  it  was  discovered  that  because  of  a  chain  of  circimstances, 
many  of  the  program  materials  could  simply  not  be  used  by  a  large  part  of  the 
target  audience  due  to  the  inaccessibility  of  equipment  and  the  problem  of 
rotating  shcedules* 


3,  Preparatory  and  follow-^up  work 

Through  interaction  with  teachers  the  ideal  means  of  preparatory  and 
follow-up  work  can  and  should  be  specified.     This  preparatory  and  follow-up 
work  should  help  to  achieve  the  program*!  objectives  and  the  objectives  of  the 
overall  curriculum.    One  of  the  only  ways  that  this  effect  can  be  tested  is 
to  contrast  the  use  of  teacher  input  with  a  comparison  group  who  do  not  have 
teacher  input  such  as  was  done  in  the  "Allons-Y"  research. 

4,  Teacher  knowledge  about  using  the  program 

Through  direct  contact  with  the  teachers,  the  evaluation  should 
determine  whether  teachers  are  able  to  use  the  programs  or  whether  concrete 
guidelines  should  be  provided.    By  having  the  teacher  view  a  number  of  programs 
and  asking  how  the  programs  would  be  used,  the  evaluation  team  should  be 
able  to  determine  whether  concrete  guidelines  are  needed  to  use  the  program. 
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5,     The  preparation  of  itudents 

The  evaluation  team  should  try  to  detemine  whether  concrete  preparation 
in  the  viewing  of  educational  television  materials  is  necessary  for  students. 
Certainly^  they  need  no  education  to  watch  television  but  they  may  need 
some  help  in  gaining  the  most  froin  an  educational  talevision  learning 
experi ence. 
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Factors  wtuLch  need  students  viewing  the  proggj^ 
1 ,  Attention 

Attention,  interest  or  appeal  of  the  program^  is  ona  of  the  first  , 
variables  to  be  tested  when  students  are  involved*    Attention  can  be  measured 
by  various  means  ranging  from  a  simple  count  of  eyes  on  screen,  to  program 
analyzer  techniques,  conjugate  reinforcement  (Lindsley,    O,  R,  1962,  Ogston  1975)^ 
aye  movement  stuuies  (O'Bryan,  1975),  distraotor  analysis  (Palmer,  1974),  or 
similar  techniques.    Lewis  (1973)  reviewed  various  means  of  measuring  attention 
to  audio^visual  materials. 

The  purpose  of  measuring  the  attention  is  to  ensure  that  each  segment 
of  the  prograin  is  interesting  enough  to  be  watched  by  ttie  students.     If  the 
student  is  not  watching  the  program,  all  future  measurement  will  be  in 
jeopardy. 

Attention  can  be  tested  rither  in  the  group  situation  or  in  a  viewing 
situation  in  which  one  student  or  a  small  gmup  watching  the  program  is 
nranitored  by  one  of  the  techniques  mentioned  above  to  increase  the  vocabulary 
level  of  students,  help  them  hear  the  spoken  language  and  help  them 
improve  their  ability  to  construct  sentence  in  the  target  language*  The 
purpose  of  tte  on-site  evaluation  is  to  determine  whether  any  vocabulary 
or  correct  construction  has  been  acquired  through  interaction  with  the 
program, 
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3.    Comprehenaion  of  the  Progran 

Often  in  the  target  language,  although  the  vocabulary  ia  at  a  level 
which  can  be  understood,  the  complexity  of  the  plot  or  other  factors  make 
it  hard  for  students  to  undarstand  the  axact  course  of  action.    As  a  rasult, 
the  romprehenaion  of  the  program  must  be  measurad.     There  are  various  procedi 
to  asaesi  comprehension,  the  almplast  one  baing  an  interview  with  the  studen 
to  determine  what  he  or  she  remembered  from  tiie  program.     Other  means  which 
could  be  used  include  a  question  tachniqua  with  multiple  alternatives,  or  a 
recall  test  in  which  students  are  simply  asked  to  write  down  what  happened. 


4,    Attitude  to  program  segments 

The  enjoyment  of  the  various  segments  or  total  program  is  also  one  of 
the  most  important  areas  to  ffleasure.    In  the  case  of  second  language  instructio 
although  the  program  may  fail  to  teach  any  vocabulary  or  construction,  or 
even  in  fact  may  fail  to  ba  conpletely  understood  ^  students,  they  may  enjoy 
the  interaction  with  tha  program.    The  attitude  to  program  sagments  also 
would  measure  attitude  to  production  variables,  level  of  plot,  plot  complexity, 
the  actors  and  any  other  relevant  information. 


5.    Attituda  to  production 

The  students  who  view  our  second  language  programs  will  have  viewed  a 
great  deal  of  television  by  the  time  they  interact  with  one  of  tha  programs 
that  we  produce.    As  a  result,  it  is  important  to  determine  whether  they 
consider  tha  production  level  to  be  similar  to  that  of  the  commercial  programs 
that  they  watch,    if  the  production  level  is  seen  to  be  inferior  to  the 
programs  they  watch,  this  will  likely  interfere  with  their  learning  or  enjoyment 
of  the  program.    The  attitude  to  production  can  ba  obtained  using  open  ended 
stateBients  or  using  Likert  ^pe  scales. 


g,     post-"prograni  monitoring 

Many  of  the  effects  of  the  second  langiiage  program  will  not  be 
evident  inmediately  after  the  program  is  viewad.    Howevar,  they  inay  wall 
be  exhibited  later  on  after  the  students  have  had  time  to  digast  the  prograin 
and  fit  it  in  to  their  regular  work.     The  purpose  of  poat-program  activity 
monitoring  is  to  determine  the  ability  of  the  program  to  produce  changes  in 
the  behavior  of  the  students.     The  usage  of  vocabulary  constructions 
learned  in  the  program,  the  repetition  of  the  types  of  activities  produced 
in  the  program  would  seem  to  indicate  whether  the  program  vrauld  be 
successful  in  promoting  long-term  change.    Post-program  monitoring  activity 
would  have  to  take  place  through  obsarvational  procedures  or  other  similar 
devices, 

7 ,    Attitude  to  French  course 

One  would  expect  that  if  one  group  of  students  is  using  a  French  television 
series  while  the  other  group  is  using  standard  curricular  materials  there 
might  be  a  difference  in  attitudes.    Gardner  and  Smythe  (1975)  have  suggested 
an  attitude  to  French  course  questionnaire  which  could  probably  be  used  to 
measure  attitudes  to  the  French  course. 

Conclusions 

The  formative  evaluation  of  bilingual  television  programs  requires  a 
multi-faceted  research  approach  so  that  the  various  aspects  of  the  programs 
can  be  examined.    No  one  approach,  such  as  field  testing,  can  provide  guidance 
to  producers  of  television  programs.    A  variety  of  methods  provide  answers  to 
the  many  questions.    Many  of  the  techniques  mentioned  above  have  been  tried 
and  tested,  both  the  two  evaluations  described  above  and  in  other  formative 
evaluations  of  television  programs.    However,  some  of  the  other  procedures 
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have  emanated  from  the  research  but  have  not  been  fully  tested.     There  is 
much  work  to  be  done  in  this  field  by  researGhers  in  the  field  of  eduaational 
technology.     The  best  research  designs  and  measurement  teGhniques  must  be 
marshalled  to  advance  the  learning  in  this  area. 


7uu 


REFERENCES 


Fleming,        G.     Evaluation  of  fom  educational  television  program  series 
produced  by  the  Ontario  Educational  Communications  Authority* 
Research  Paper  No.  26.    Torontoi    0*E*C,A,  1972, 

Friedlander^  D.  Maximizing  comprehension  of  informational  films  and 

videotape:     foOTative  evaluation  during  production,    Mimeo,     New  ^ 
England  Instructional  Television  Research  Center,  University  of 
Hartford,  Hartford,  Connecticut*     1974 . 

Gardner,  R,  C. ,  Ginsberg,        B.  and  Snathe,        C,      Attitudes  and  motivation 
in  second  language  learning:     a  social  psychological  perspective - 
The  Canadian  Modern  Language  Reviavr^     1976    J2  243ff* 

Gardner,  R.  C.  and  Smyths,  P.  C*     Second  language  acquisitiom     a  social 
psychological  approach.     Research  Bulletin  232,  Department  of 
Psychology,  University  of  Western  Ontario,  London,  Ontario,  1975. 

Johns  ton-Doyle,  K.  and  Lewis,  R,F.    k  content  analysis  of  student  comments 

on  "Le  masque  de  Marie  Louise"  i    A  program  in  the  "Allons-Y"  series. 
Halifax,  mi    Atlantic  Institute  of  Education  R.  &  D,  Bulletin 
No.  1,  1978. 

Laosa,  Luis,  M.    Formative  research  and  evaluation  in  the  application  of 

talevision  madia  to  bilingual  multicultural  education.  California 
Journal  of  Educational  Research,  1975,     25     (5)  302-309. 

Lewis,  Richard  F.     An  evaluation  of  "Allons-Y'*    Halifax,  NS:     Atlantic  Institute 
of  Education.    Technical  report,  1979, 

Lewis,  Richard  F.    The  effect  of  stmulus  color  and  degree  of  realism  on 

attention,    Syracuse  University,  Syracuse,  New  York,  unpublished 
doctoral  dissertation,  1973. 

L^wis    R    P    and  Fisher,  G.      An  evaluation  of  the  junioi^  high  school  series 
-""''^  U^lLf^,  Atlantic  institute  of  Education. 

Technical  report,  1977. 

Lindsley,    0.  R.    A  behavioral  measure  of  television  viewing.    Journal  of 
Advertising  Reseageh,  1962,    £  2-12. 


ERIC 


O' Bryan,  K.  G.  Television  and  eye  inoveinent  resaarch.  Mlmeo. 
Edueational  Cominuni cations  Authority,  1975. 


Toronto;  Ontario 


680 


Ogston,  D.  G.    An  inetructional  television  behavioral  pretest  measure  and  its 
relatiDnslu^p  to  learning.     Jo^nal  of  EducatiQnal  Tschnology 
gystemi ,   1975    4     (2),  125-238. 

Pa.lmer,  E.  L,    Formative  research  in  the  productiQn  of  television  for  children. 
Chapter  12  of  the  73rd,    Yearbook  of  the  National  Society  for  the 
Study  of  EdUGation.    Chicago,  IllinQiai    National  Society  for  the 
Study  of  Education,  1974, 

Williams,       ,  Van  Wart,  G,   and  Stanford,  M,  National  evaluation  of  a 

Spanish/English  educational  television  series,     Austin,  TeKasi 
Center  for  Cormiuni cation  Research,  University  of  Texas,  1973. 


681 


EKLC 


EDUCATIONAL  DEVELOPMENT:    THE  REPUBLIC  OF  KOREA 

(1970-1979) 

by 

Robert  M,  Morgan 


February,  1979 


LEARNING  SYSTEMS  INSTITUTE 
Florida  State  University 

Tallahassee,  Florida  32306 


Presented  at  AECT, 
March  6,  1979, 
in  New  Orleans 


ERIC 


7uG 


682 


ABSTRACT 


In  an  effort  to  pro vide  a  better  education  for  more  of  its  citizens 
the  Korean  government  has  undertaken  a  number  of  steps  since  1970.  These 
Included  a  major  systems  study  of  the  nations  educational  progranis  in  1970-71  . 
This  study  provided  the  basis  for  a  major  development  effort  and  reform  of  the 
elementary-middle  school  system.    In  1972  a  national  research  capability-- the 
Korean  Educational  Development  Insti tute— was  created  to  develop  and  enipirlcally 
validate  the  proposed  new  system.    Although  there  have  been  a  nuitiber  of 
changes  from  the  system  as  1t  was  originally  concalveds  KEDI  has  been  at 
work  building  the  new  program  for  the  past  seven  years.    The  latest  large 
scale  field  tryouts  (231,000  students  in  1978)  indicate  that  the  new  system— 
largely  buHt  on  Instructional  system  design  concepts--is  significantly  more 
effective  than  the  traditional  elementary-middle  school  programs,  and  not 
more  expensive. 
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EDUCATIONAL  DEVELOPMENT:    THE  REPUBLIC  OF  KOREA 

(1970-1979) 
Robert  M,  Morgan 

A  Sj'Stems  Study  of  Korean  Education  -  1970 

The  major  aim  of  this  study  was  an  attempt  by  the  Republic  of  Korea  to 
find  if  it  might  be  able  to  organize  its  educational  fesources  in  ways  that 
would  make  its  educational  programs  more  responsive  to  the  nation's  needs 
and,  simultaneously,  function  more  efficiently  than  its  present  educational 
system.    The  general  problem  could  only  be  addressed  by  answering  a  large 
number  of  corollary  questions  and  the  Korean  government  invited  the  Florida 
State  University  to  mount  a  project  for  that  purpose.    In  the  planning  phase 
of  the  project  it  was  judged  that  a  "systems  approach"  to  the  analysis  of 
Korea's  educational  sector  would  be  more  suitable  and  a  range  of  expertise 
and  competence  was  identified  as  critical  to  the  survey  and  analysis  process. 
It  was  quickly  apparent  that  the  varieties  of  needed  competencies  would 
require  an  interdisciplinary  team  of  specialists  and  the  University  assembled 
a  study  team  of  seven  persons.    The  study  team  Included  (1)  an  economist; 
(2)  a  manpower  specialisti  (3)  an  educational  administrator/manager-,  (4)  an 
educational  technologist;  (5)  a  teacher  training  specialist;  (6)  a  systenis 
management  specialist;  and  (7)  a  behavioral  scientist.    The  project  also 
utilized  several  American  and  Korean  consultants  as  they  were  needed.  The 
economist,  a  graduate  of  Seoul  National  University,  was  also  a  native  Korean. 

The  study  team  spent  three  months  in  the  Republic  of  Korea  in  1970 
gathering  information  and  data  about  the  educational  system,  the  economy, 
the  nation's  needs  and  wants  for  its  educational  programs  and  the  resources 
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available  for  potential  improvement  of  the  system.   Prior  to  its  arrival  in 
Korea,  the  study  team  had  Identified  several  areas  in  which  informrUion  m$ 
needed  and  the  Central  Educational  Research  Institute  (CERI)  In  Seoul  assembled 
an  Gxtensive  Inventory  of  historical  data,  research  reports,  plahning  docuinents 
and  other  relevant  material.    This  preliminary  work  by  CERI  proved  to  be  a 
great  timesaver  and  added  significantly  to  the  efficiency  of  the  study  tearn. 
The  early  portion  of  the  team's  stay  in  Korea  was  devoted  to  identifying 
additional  information  requirements  and  more  carefully  formulating  the  questions 
that  needed  answers.    CERI  and  the  Ministry  of  Education  then  worked  with  the 
team  in  identifying  the  most  appropriate  sources  of  Information  and  answers 
to  the  questions.    At  times  during  the  period  In  Korea  the  study  teani  operated 
as  a  whole,  but  inore  often  the  individual  members  of  the  team  worked  separately 
with  counterpart  Korean  specialists.    Before  the  three  months  were  over  members 
of  the  study  team  had  visited  schools  at  all  levels  throughout  Korea  and  had 
talked  to  hundreds  of  teachers,  school  administrators  and  students.   The  team 
also  worked  closely  with  several  Korean  government  agencies,  most  particularly 
the  Ministry  of  Education  (MOE)  and  the  Economic  Planning  Board  (EPB).    By  the 
end  of  the  team's  stay  in  Korea  a  great  deal  of  information  had  been  gathered, 
which  the  team  then  began  to  sift,  organize  and  analyze. 

Clearly,  the  study  team  could  not  in  the  few  months  available  become  In- 
depth  experts  In  the  social,  econwnic  and  educational  affairs  of  the  Republic 
of  Korea,   However,  a  great  deal  was  learned.    From  what  was  learned  the  study 
team  formed  certain  impressions,  developed  conclusions  and  made  some  rather 
sweeping  recOTiTiendations  for  educational  reform  in  Korea.   The  team  recognized 
that  there  was  a  danger  of  incompleteness  and  even  inaccuracy  in  so  hurried  an 
Infonnatlon  collection,  but  judged  the  available  data  of  sufficient  validity 
to  justify  its  conclusions  and  recommendations.    There  were  some  errors  or 
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data  gaps  in  the  study  that  were  apparent  to  the  experienced  Korean  educator 
when  they  reviewed  the  study.    Korean  government  officials,  educators  and 
researchers  determined  that  these  errors  did  not  significantly  effect  the 
conclusions  and  recommendations. 

In  the  study  and  analysis,  focus  was  on  those  Issues  which  would  help 
the  Korean  Republic  provide  a  better,  more  relevant  education  for  more  Korean 
young  people  at  a  lower  unit  cost  and  at  a  total  cost  not  greater  than  the 
nation  could  afford.    To  this  end,  the  study  team  collected  historical, 
cultural  and  educational  data,  including   demographic  reports,  economic 
forecasts,  manpower  needs  projections,  educational  fiscal  data,  current  and 
long-range  educational  plans  and  such  Information  as  was  available  on  educa- 
tional  objectives  and  attainment.    The  data  was  analyzed  in  terms  of  future 
manpower  needs  and  educational  output,  estimated  cost  benefits,  and  strategies 
for  appropriate  introduction  of  innovation  and  technology  into  the  educational 
system.   Alternative  approaches  to  relating  resources  to  educational  objectives 
and  problems  were  examined. 

EcononilC  Factors 

* 

In  the  post-Korean  Conflict  period,  the  Korean  economy  had  txperienced 
remarkable  industrial  progress  and  growth.   There  seemed  to  be  general  agree- 
ment among  economists,  both  Korean  and  foreign,  that  the  economy  would  continue 
to  grow  in  the  foreseeable  future,  would  become  somewhat  more  stable  and  would 
lead  to  an  Improved  balance  of  payments  position.    The  labor  force  was  inceas- 
ing  steadily  and  the  rate  of  unemployment  was  decreasing.    All  indications 
were  that  1n  general  business  would  be  good  but  in  an  economy  as  dynamic  as 
that  as  Korea,  economists  did  not  appear  to  be  able  to  predict  very  many  years 
in  the  future  nor  with  much  specificity  what  that  business  would  consist  of. 


It  seemed  unlikely  that  manpower  needs  forecasts  that  extended  more  than  two 
or  three  years  Into  the  future  would  be  of  much  val idlty— particularly  at  the 
technical  and  subprofesslonal  levels.    Yet,  1t  was  at  these  levels  that  Korea 
was  likely  to  develop  its  greatest  manpower  deficiencies  which  would  be 
qualitative  rather  than  quantitative.    At  that  time  In  Korea,  it  was  not 
unusual  for  a  job  holder  to  have  an  education  which  was  largely  unrelated  to 
the  particular  job  functions  performed.    Furthermore,  manpower  waste  through 
over  qualification  of  Incumbent  job  holders  could  be' observed  and  was  antici- 
pated to  Increase.    A  major  problem  was  likely  to  result  In  the  future  from 
the  lack  of  congruence  between  the  nation's  manpower  requirements  and  the 
projected  supply  of  skilled  labor,  subprofesslonal s  and  technicians.    It  was 
suggested  that  the  only  long  range  solution  to  these  problems  was  a  reordering 
of  the  educational  priorities  in  the  schools  of  Korea. 

In  recent  years,  the  burden  of  financing  public  education  in  Korea  had 
increasingly  moved  to  the  federal  government  and  away  from  the  private  citizen, 
though  the  private  citizen  was  still  a  heavy  contributor.   The  formal  educa- 
tional system  of  Korea  had  become,  to  a  large  extent,  a  publicly  controlled 
service.    It  was  noted  that  purely  in  terms  of  economic  needs,  the  quantity 
of  schools  may  be  sufficient  for  the  next  few  years,  but  the  quality  of  human 
resources  produced  by  existing  middle  and  high  schools  can  reach  neither  the 
level  for  which  purpose  these  Institutions  are  intended  nor  that  required  for 
the  nation's  employnient  needs.   The  study  team  contended  that  investment  1n 
education  is  as  important  to  economic  development  as  investments  In  physical 
capital.    As  investment  in  education  competed  with  investment  in  physical 
capital  In  the  allocation  of  scarce  national  resources.  It  was  prudent  for 
Korea  to  Invest  relatively  more  in  the  middle  schools  than  in  the  higher  levels 
of  the  educational  system.   Two  rate  of  return  studies  were  undertaken  in  this 


project,  which  indicated  that  for  the  high  school  levels  the  rate  of  return 
{11,2$)  was  not  much  different  than  that  for  the  college  and  university 
graduates  (9*5^),    The  rate  of  return  for  the  middle  school  education,  how- 
ever, was  noticieably  higher  (20-0%)»    This  latter  rate  of  return  was,  1n 
fact,  higher  than  that  for  the  average  capital  investment  In  Korea  at  that 
time. 

There  were  important  Implications  of  these  economic  and  manpower  consid- 
erations for  governmental  and  educational  decision-makers  In  Korea.  The 
greatest  needs  for  manpower  in  the  years  ahead  would  be  at  the  level  of  the 
middle  school  graduate.    The  rate  of  return  on  Investment  was  astonishingly 
high  at  this  level.   The  educational  programs  through  the  elementary  and 
middle  school  were  not  as  appropriate  to  the  future  economic  needs  of  Korea 
as  they  could  be.    Using  economic  criterion  such  as  earnings,  employment  and 
maxImlEing  econoniic  benefits,  one  concluded  that  the  expansion  and  improvement 
of  the  elementary  and  middle  school  programs  should  have  been  given  high 
priority*    There  were  also  social  and  humane  arguments  to  support  this  conten^ 
tion* 

The  Contemporary  Korean  School  System 

The  educational  goals  that  characterized  the  Korean  elementary  and 
middle  schools  appearad  to  be  restricted  to  the  conventional  acadmic  domain. 
The  student  learning  outcofties  at  these  levels  fell  almost  exclusively  Into 
the  informational  and  skill  categories  of  educational  objectives*  Students 
were  acquiring  the  skills  of  readings  writing  and  computatlonp  though  with 
variable  proficiency.    This  system  seemed  to  be  characterized  by  rote 
mefnorlzation  of  classically  academic  subjects  with  the  overriding  objective 
being  to  prepare  the  student  for  the  national  competitive  examinations. 


These  exams  were  used  to  select  those  students  who  would  be  pennltted  to  enter 
the  next  level  of  education.   The  exam  for  entrance  to  the  mldclle  school  had 
been  eliminated  recently,  but  the  study  team  did  not  see  evidence  that  this 
action  was  having  any  positive  effects  on  the  curriculum.    The  existing  curri- 
culum was  not  as  relevant  to  preparing  Korean  children  to  live  and  prosper  as 
adults  as  It  could  have  and  should  have  been  .    The  study  team  did  not  attempt 
to  specify  educational  objectives.   This  could  only  be  done  by  Korean  educators. 
However^  the  team  felt  the  curriculum  could  be  broadened  to  Include  the  teaching 
of  inquiry  skills,  problem  solving  approaches  and  generally  attend  more  to  pro- 
cess objectives— and  that  these  should  not  only  be  learning  outcomes,  they  could 
also  serve  as  effective  Instructional  means.    A  general  addition  to  the  elemeri- 
tary-middle  school  curriculum  and  important  to  the  enhancement  of  Its  relevance, 
would  be  the  addition  of  preoccupatlonal  education.    It  was  suggested  that  a  - 
properly  conceived  preoccupatlonal  program  would  add  to  the  graduate's  employ- 
ability,  his  retrainablllty  and  his  occupational  mobility.    In  other  words, 
the  products  of  nine  years  of  education  in  Korea  could  grow  into  a  valuable 
Inventory  of  manpower  which  with  limited  but  specific  additional  training 
could  be  prepared  for  technical  and  skilled  occupations  as  these  needs  develop 
and  change. 

The  study  team  predicted  that  the  present  Korean  educational  system  could 
not  In  its  existing  form  achieve  these  Important  objectives  through  simple 
expansion  or  minor  alteration  of  the  existing  system.   An  additional  problem 
was  that  with  6.7  million  children  In  school  in  the  age  range  of  six  to  four- 
teen years,  nearly  a  million  youngsters  were  out  of  school.    By  1975  there 
would  have  been  8.2  million  Korean  children  in  the  eligible  age  range  for  the 
elementary  and  middle  schools.    There  are  many  reasons—manpower  needs,  societal 
stability  and  humanitarian— that  led  the  study  team  to  conclude  that  it  was 


essential  to  expand  free,  universal  education  to  include  the  middle  school 
level.    It  was  noted  that  free  education  as  it  is  understood  in  the  United 
States  did  not  exist  in  Korea  except  for  a  few,  very  poor  families.  Almost 
all  families  contributed  directly,  through  purchase  of  textbooks,  tuition 
and  various  fees,  to  the  cost  of  educating  their  children.    When  these  hidden 
contributions  to  school  operations  were  added  to  the  visible  tax  contribution, 
the  per  student  per  year  cost  was  estimated  to  average  12,878  Won  for  the 
elementary  and  middle  schools  combined,^ 

A  Proposed  New  Educational  Model  -  1971 

In  order  to  develop  a  nation  of  people,  all  of  whom  had  been  prepared 
for  a  life  of  fulfillment  in  terms  of  general  occupational  and  citizenship 
education,  the  study  team  suggested  in  Its  report  that  a  nine-year»  free  and 
compulsory  educational  program  was  necessary.    If  the  public  schools  through 
the  ninth  year  were  open  to  all  students,  were  free  and  of  uniformly  high 
quality,  it  was  reasonable  to  assume  that  the  need  for  private  schools  and 
out-of -school  tutoring  would  virtually  disappear.    It  was  also  reasonable  to 
assume  that  funds  used  for  these  purposes  would  be  diverted  to  public  school 
support,  though  In  the  form  of  public  school  taxation. 

The  vocational  high  schools  of  Korea,  which  enrolled  slightly  more  than 
half  of  those  students  permitted  to  enter  high  school  following  graduation 
from  the  middle  schools,  were  not,  1n  the  Judgement  of  the  study  team,  effec- 
tively serving  the  purposes  for  which  they  were  formed.    Based  upon  assumptions 
about  the  potential  for  improved  academic  accomplishment  at  the  elementary 

-  All  Won  figures  in  this  report  are  based  on  1969  values.    The  exchange 
rate  at  that  time  was  approximately  370  Won  to  the  U.S.  donar. 


and  middle  school  level,  the  study  team  recommended  that  responsibility  for 
all  post-ninth  grade  occupational  training  be  consoHdatad  under  a  single 
governmental  agency  and  that  this  training  should  have  been  directed  exclu- 
sively to  preparing  people  for  specific  jobs.    These  job  training  programs 
would  have  been  of  variable  duration  depending  on  the  training  requirements, 
would  have  been  operated  only  as  long  as  there  were  known  manpower  needs  for 
the  Jobs  in  question,  and  would  be  open  to  qualified  citizens  of  any  age  level. 
It  was  suggested  that  the  vocational  high  schools  of  Korea  should  become  an 
integral  part  of  the  job  training  program  and  cease  to  operate  in  their  present 
fomi. 

The  study  team  recoimiended  that  the  Korean  colleges  and  universities  and 
the  academic  high  schools  that  feed  them  be  maintained  at  their  present  rate 
of  growth  and  Improvement  measures  of  an  evolutionary  and  gradual  character 
be  undertaken  in  the  future.    These  Improvement  measures  would  be  of  the  kind 
ordinarily  expected  in  the  normal  course  of  events.    Based  on  projections  of 
the  Ministry  of  Science  and  Technology  for  manpower  needs  at  the  higher  levels- 
scientists,  engineers,  professional  managers— the  Ministry  of  Education  would 
need  to  reappraise  the  enrollment  quotas  for  the  various  subject  areas  In  the 
universities. 

The  highest  priorities  for  extraordinary  change  and  development  for  Korea 
should  have  been  at  the  elementary -middle  school  level.    It  was  believed  that 
through  a  substantial,  but  feasible,  effort  In  the  development  and  validation 
of  a  significantly  different  kind  of  elementary-middle  school  that  Korea  could 
provide  an  educational  program  of  demonstrably  higher  quality  and  relevance 
for  all  age-eligible  Korean  youngsters.    Further,  it  was  predicted  that  this 
program  once  developed  and  Installed  in  the  nation's  schools  would  not  only 
be  cost  effective,  it  would  In  fact  cost  less  per  student  to  operate  than  was 


presently  the  case* 

The  new  school  proposed  by  the  study  team  involved  a  number  of  changes 
from  the  eKisting  system*    These  Included  changing  the  basic  instructional 
unit  from  its  present  class  size  form  to  a  larger  grouping^  Introducing 
individualized  Instructional  cohcepts  and  associated  mater^ialSi  modifying 
the  role  of  the  teaching  staff  and  increasing  the  ratio  of  students  to 
teacherSs  and  using  programned  instructional  television  and  radio. 

It  was  proposed  that  the  students  would  be  organized  into  "instructional 
units"  of  300  students  with  the  average  sized  Korean  school  having  three  such 
units.    Each  instructional  unit  would  become  the  responsiblity  of  a  four 
person  teaching  team  whose  functions  would  be  differentiated  and  carefuny 
defined  in  terms  of  what  each  team  member  contributed  to  the  learning  experi- 
ence of  the  students.    This  would  raise  the  students-teacher  ratio  from  the 
present  55-1  to  75-1.    It  was  proposed  that  the  instructional  unit  (with  300 
students  and  four  staff  members)  would  have  permanently  assigned  to  it  six 
conventional  classrooms.    This  would  make  an  average  student-to--classroom 
ratio  of  50-1,  down  from  the  present  66-1.    In  order  to  get  the  learning 
group  into  sufficient  space»  the  Korean  government  would  face  a  socially  and 
politically  difficult  decisions  that  of  moving  their  schools  to  a  double 
shift  basis.    Because  of  the  self -study  nature  of  much  of  the  planned  educa- 
tional materials^  the  students  would  have  been  able  to  do  more  learning  in 
their  homes  or  out  of  school.    It  should  have  been  possible  to  shorten  the 
time  in  school  for  students  without  reducing  real  Instructional  time  or 
learning  achievement. 

A  basic  recoimiendation  of  the  study  team  was  that  the  Korean  elOTentary- 
middle  schools  be  moved  to  a  system  of  Individualized  instruction.    The  intro- 
duction of  an  individualized  approach  should  have  several  benefits.    It  was 
performance  based,  permitted  students  to  move  at  their  own  learning  rate. 


and  placed  a  larger  measure  of  responsibility  on  the  student  for  self-direction 
of  his  learning  experiences.    It  also  reduced  reliance  on  direct  teacher-to- 
student  Instruction.    The  basic  instructional  resource  for  that  portion  of 
the  curriculum  to  be  Individualized  .was  a  "student-learning  unit,"  which  was 
prepared  in  modular  and  overlapping  form  and  was  packaged  for  ease  of  storage 
and  retrieval  by  students.    These  units  should  have  been  sufficiently  durable 
to  have  a  use-life  of  four  to  five  years.    The  student-learning  unit  contained 
the  behavioral  objectives  for  the  unit,  critical  instructional  materials  and 
directions  to  other  resources  not  contained  in  the  package,  and  formative 
criterion-referenced  test  Items  which  pennltted  the  student  to  assess  his  own 
progress  through  the  unit.   The  principles  of  programmed  instruction  were 
eniployed  in  the  development  of  these  units  even  though  most  of  the  instruc- 
tional materials  were  not  progranined  instruction  per  se. 

Another  feature  of  the  proposed  program  was  that  the  teaching  staff  should 
be  differentiated  In  a  manner  that  calls  for  professional  staff  with  differen- 
tiated specialties.    This  provided  a  better  means  for  having  the  full  range 
of  competencies  available  In  the  instructional  unit  and  made  It  possible  to 
allocate  different  responsibilities  to  the  individual  professionals  making 
up  the  team.   The  team  functions  must  derive  from  an  empirical  analysis  of 
the  new  learning  program  and  required  special  training  be  given  to  the  teach- 
ing team.   The  teaching  team  would  operate  under  the  direction  of  a  master 
teacher  whose  main  job  was  the  management  of  the  learning  environment. 

The  study  team  proposed  that  a  national  educational  radio  and  television 
distribution  system  be  developed  which  would  continuously  transmit  instruc- 
tional programs  during  the  school  day.    It  was  estimated  that  one  and  one- 
half  to  two  hours  of  television  instruction  would  be  received  by  each  student 
each  day.  comprising  about  one-third  of  the  student's  instructional  day. 
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The  type  of  television  suggested  was  one  which  couples  the  principles  of 
programmed  Instruction  and  good  dramatic  television  production  to  yield  pro- 
grams which  are  interesting  and  will  teach  youngsters  who  are  widely  varied 
In  age  and  socio-economic  background.   Television  sets  could  be  made  available, 
progranimlng  and  maintenance  capability  existed,  and  a  working  prototype  for 
central  transmission  and  nationwide  relay  was  in  place.    It  was  estimated 
that  a  functional  national  educational  television  systetn  could  be  built  and 
Installed  including  a  television  set  for  each  Instructional  unit  of  300 
students  plus  an  Inventory  of  replacement  sets  for  maintenance  rotation. 
This  television  system  would  be  an  integral  component  of  the  system  of  In- 
structional resources  and  would  not  be  an  "add-on"  to  the  existing  instruc- 
tional program.    It  would  be  a  form  of  programned  Instruction  developed  to 
teach  specific  behaviors  and  would  call  for  active  responses  from  the  student. 
Auxinary  printed  materials  would  be  developed  to  go  with  the  ITV  programs  in 
which  the  students  would  write  responses,  solve  problems  and  record  reactions 
and  questions.    Student  learning  would  be  closely  monitored  and  the  teacher 
would  be  furnished  supportive  and  supplementary  materials  to  help  her  work 
individually  with  any  students  who  experience  difficulty  or  who  fall  behind 
In  the  televised  Instruction. 

In  the  proposed  Korean  elementary-middle  school  it  was  anticipated  that 
radio  instruction  would  be  used  in  the  context  of  the  Individualized  program 
and  be  one  of  the  instructional  resources  to  which  the  student  was  directed. 

These  were  extensive  changes  recommended  by  the  study  team  in  the 
educational  processes  for  the  Korean  elementary  and  middle  schools.  Deciding 
upon  the  appropriate  educational  goals  and  operationally  defining  them  into 
specific  Instructional  objectives  was  a  task  of  enormous  importance  to  the 
future  of  Korea,    The  kind  of  Individualized  program  being  reconmended  by  the 
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by  the  study  team  would  work  best  1f  continuous  progress  of  each  student  was 
pemiitted. 

Vocational  Education 

The  part  of  education  about  which  Korean  leaders,  both  governmental 
and  educational,  invariably  expressed  the  greatest  concern  was  vocational 
preparation.    The  study  team  strongly  recomnended  the  addition,  at  both  the 
prlniary  and  middle  school  levils,  of  a  substantial  offering  in  pre-occupa- 
tional  pfeparatlon.   The  learning  of  the  specific  technical  job  skills  needed 
in  Korea's  economy  could  then  be  readily  acquired  on  a  minimum  training  time 
basis.   The  graduates  of  the  proposed  nine  year  curriculum  should  have  solid 
academic  preparation  and  should  also  be  well  grounded  In  the  general  funda- 
mentals and  prerequisites  to  specific  job  training.    These  changes  will  make 
the  vocational -technical  high  schools  as  they  have  traditionally  operated 
obsolete  and  unnecessary. 

What  would  be  needed  was  a  system  of  job  training  programs  which  would 
have  the  following  characteristics: 

A.  The  training  would  be  exclusively  related  to  specific  jobs  that 
would  be  avail  able  as  students  complete  the  training.  Manpower 
needs  forecasting  would  be  essential  for  these  purposes. 

B.  The  program,  growing  out  of  short  range  (two  years  or  Less) 
manpower  needs  predictions,  would  be  of  variable  duration,  the 
training  being  no  longer  than  is  minimally  required  to  prepare 
the  trainees  for  the  available  specific  Job.   These  programs 
would  vary  from  four  weeks  to  two  years  in  length. 

C.  The  system  would  provide  for  the  start  up  of  new  training  programs, 
with  the  shortest  possible  lead  time,  as  needs  change.    By  the 


same  token »  programs  would  be  terminated  as  they  were  no  longer 
needed . 

D.  These  programs  would  be  staffed  with  personnel  who  knew  the  Job 
skills  being  taught.  With  m^ch  less  emphasis  on  degrees,  teacher 
certification  or  other  formal  educational  requirements. 

E.  These  schools  would  not  only  train  middle  school  graduates;  they 
would  be  used  for  retraining  of  adult  employees  as  peRsonnel  needs 
change.         .     .  ' 

F.  There  would  need  to  be  a  very  close  liaison  and  coordination 
between  the  appropriate  governmental  agencies  and  the  private 
sector  to  maintain  the  optimum  relationships  between  training 
output  and  manpower  utilization. 

Evaluation 

To  optimize  the  adoption,  effect  and  continued  Improvement  of  these 
innovations,  it  would  be  necessary  to  develop  an  efficient  appraisal  and 
evaluation  activity  which  would  provide  policy-makers  and  public  in  general 
with  Information  about  the  achievements,  and  problems  which  resulted  from 
these  innovations.    Information  for  this  evaluative  function  should  be 
derived  from  the  performance  of  individual  students  and  various  groupings  of 
students,  and  at  the  component  level,  it  must  come  from  the  performance  of 
the  various  educational  components  which  comprise  the  Innovative  pattern 
being  proposed.    Only  by  covering  the  range  of  information  from  the  student 
to  the  ccMflponent  would  the  public  and  educational  decision-makers  be  able  to 
know  the  effectiveness  with  which  the  educational  system  was  performing. 

The  purpose  of  evaluation  was  to  provide  Information  which  would  permit 
the  continuing  Improvement  of  the  educational  process.    Its  scope  should  be 
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system-wide^  with  a  coitiprehenslve  role  for  evaluation  at   each  level  ^  ranging 
from  tracking  the  progress  of  the  individual  student  to  assessing  the  perfor- 
mance of  the  total  system  on  a  national  scale.    Evaluation  should  be  based  on 
systems  perforrnance  with  respect  to  the  defined  goals  and  objectives  for  Korean 
education.    With  educatfon^ conceptualized  as  an  Input-process-output  model,  the 
data  base  for  the  evaluation  systeni  would  be  described  In  terms  of  the  corre- 
sponding sets  of  data  comprehensive  evaluation  of  the  developmental  effort, 
therefore^  would  provide  descriptive  and  diagnostic  feedback  for  the  planning 
and  developments  Implementation^  operatlont  and  diffusion  levels.  Emphasis 
would  be  placed  on  evaluation  at  the  student  level ^  the  component  level t  the 
school  level  and  the  system-wide  or  national  level.    The  basic  purpose  of  this 
evaluation  would  be  to  provide  adequates  valid  and  reliable  information  on 
performance  assessnient  at  each  level  so  that  the  highest  perforniance  possible 
may  be  reached  under  the  existing  constraints.   To  create  such  a  comprehenslva 
evaluation  systeni  required  a  series  of  activities.    The  first  of  these  was  a 
thorough  review  of  the  nature  and  intent  of  the  evaluative  systems  currently 
In  operation  In  the  Korean  educational  system-   The  second  was  to  expand  the 
evaluation  model  outlined  In  the  study  teim's  report  into  a  detailed  opera- 
tional plan  which  is  appropriate  to  Korean  education,  to  the  innovations  being 
Implemented*  and  to  the  fgnnatlve  requirements  of  these.    The  third  step  was 
the  general  process  of  testing  and  validation  of  the  evaluation  model.  The 
fourth  step  was  widespread  Implementation  of  the  evaluative  procedures.  This 
evaluation  system  would  help  bridge  the  transition  between  the  current  system 
and  the  Installation  of  the  proposed  system  and  facilitate  1t. 


Divelopment  and  Implementation  -  1972 

Korean  educational  leaders  concluded  that  the  kind  of  elementarj^--niiddle 
school  proposed  by  the  study  team  was  sensible  for  the  nation  and  appeared  to 
be  a  desirable  and  viable  alternative  to  the  present  educational  condition^ 
They  have  had  to  address  several  other  important  questions.    The  responsible 
governmant  officials  have  had  to  Judge  whether  or  not  the  proposals  that 
appear  to  be  good  for  Korea  are  also  possible.    Are  the  various  resources— 
people p  money  and  time— available?   What  are  these  resources?   How  long  would 
It  take  to  develop  such  a  system?   How  much  would  It  cost  to  develops  how 
much  to  Install  nationwide:,  then  how  much  to  operate  on  a  yearly  basis? 

The  study  team  was  optimistic  that  the  key  resource  was  avallabla*  This 
resource  was  a  group  of  aggressivep  technlcalljr  sophisticated  Korean  aduca- 
tional  researchers  who  were  prepared  to  spearhead  the  effort.  Additional 
support  and  technical  staff  needed  to  be  prepared  and  this  appMred  to  be 
feasible*   These  resource  personnel  needed  an  organlwtlon,  under  the  Minister 
of  Education p  which  would  be  funded  and  mandated  to  undertake  the  development 
and  validation  of  the  new  system.   The  study  team  proposed  an  organization 
which  It  labeled  the  Korean  Educational  Developnient  Institute  (KEDI)  whose 
responsibilities  shoutd  Include  the  design^and  tryout  of  the  system  and  its 
components.    With  personnel  frOTi  the  MOE  and  rapfiientative  Korean  educators 
KEDI  should  reappraise  the  educational  goals  ird  objectives  for  the  elementary- 
middle  schools.    It  should  develop  definitions  of  desired  learning  outcomes  at 
the  various  levels  and  then  design  and  build  the  instructional  prograins  to 
achieve  these  outcomes.    These  instructional  resources--studint  learning  units, 
ITV,  radio  and  teacher  directed  activlties^-should  be  chosen  In  tenns  of  their 
appropriateness  to  particular  content  and  objectives  In  the  curriculum.  KEDI 
should  be  responsible  for  OTplrically  danonstrating  the  instructional  effec- 
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tiveness  of  these  new  programs  of  learning,  and  should  plan  and  develop  a 
comprehensive  educational  evaluation  system.   The. evaluation  system  should 
provide  for  assigning  responsibnity  for  student  learning  to  the  principal 
elemants  in  the  educational  program,  should  provide  for  periodic  audit  of 
perforniance,  and  should  permit  system  accountability. 

The  study  team  recommended  that  a  three  phase  effort  be  undertaken: 
(1)  development-,  (2)  tryout  and  revision  1n  a  pilot  comnunltyi  and  (3) 
nationwide  d1ssen1nat1on.    It  was  imagined  that  KEDI  could  best  achieve  the 
first  two  phases  as  well  as  continued  improvenient  of  the  program  through 
research,  while  the  operating  bureaus  of  MOE  were  best  equipped  to  take 
responsibility  for  the  third  phase. 

The  recommendation  was  made  to  build  and  try  out  the  new  educational 
systeni  In  a  single  province  but  the  developers  should  keep  in  mind  that 
eventually  the  system  would  be  used  nationwide  and  the^  were  designing  to 
that  end.    Estimates  were  that  it  would  take  approximately  four  to  five  years 
to  build  and  install  the  new  system  in  a  pilot  province.    Nationwide  diffusion 
during  the  development  phase  could  probably  be  accanpHshed  in  the  same  length 
of  time,  if  the  decision  to  go  nationwide  had  been  made  at  the  onset  of  the 
program.    However,  by  deciding  to  take  the  program  nationwide  only  after  the 
pilot  system  became  operational  (as  the  study  team  reconitiended) ,  then  an 
additional  one  to  two  years  were  required  for  national  diffusion.   The  cost 
of  development  and  installation  on  a  national  scale  was  istimated  to  be  approx 
Imately  $17»000,000.00,  while  the  same  program  Installed  only  in  a  single  test 
province  was  estimated  to  cost  approximately  $7,500,000. 

The  per  student  annual  educational  cost  for  the  new  system  Was  estimated 
to  be  9,819  Won.   This  would  require  an  annual  educational  budget  for  the 
Korean  elementary-middle  schools  of  80.5  billion  Won  if  the  8.2  million  age- 


eligible  Korean  children  were  all  enrolled  in  the  public  schools  in  1975. 
This  is  contrasted  with  the  104.9  billion  Won  required  for  the  same  number 
of  children  at  the  1969  per  student  expenditure  rate. 

The  Korean  Education  Reform  1971  -  1977 

The  study  team  report  with  the  recomnindatlons  summarized  In  the  previous 
section  was  delivered  to  the  Korean  government  in  February  of  1971,  and  seven 
years  have  elapsed  since  then.   Although  work  on  the  proposed  educational 
refonn  is  still  1n  progress  enough  has  been  accomplished  to  Justify  a  sunmary 
of  the  activities  to  date. 

The  report  was  Immediately  submitted  by  the  Korean  government  to  intensfy 
and  critical  review  by  key  staff  members  of  the  Ministry  of  Education,  the 
Economic  Planning  Board  and  educational  leaders  outside  the  government. 
Robert  Morgan,  the  FSU  study  team  leader,  returned  to  Korea  in  April,  1971, 
to  participate  in  this  review  process.   As  might  be  expected  from  such  an 
exhaustive  analysis  a  number  of  errors  made  by  the  study  team  were  revealed 
and  some  conditions  were  noted  to  which  the  team  had  not  given  appropriate 
weights   However,  these  problems  were  not  thought  to  be  sufficient  to  invali- 
date the  essential  recommendations  of  the  report.    Following  the  reviews, 
with  reactions  both  pro  and  con,  the  Korean  government  decided  to  undertake 

the  refomi  program. 

In  terms  of  the  Korean  educational  program  the  report  included  two  broad 
targets  of  reform:    the  elementary-middle  school  program  (the  grades  one 
through  nine),  and  the  post-ninth  grade  vocational  educational  program. 

The  decision  was  made  to  undertake  the  revision  of  the  elementary-inlddle 
school  programs  at  once  deferring  until  later,  action  on  the  vocational 
educational  programs.   Clearly,  this  was  a  step  of  enormous  Importance  to 


the  Republic  of  Korea  and  one  which  would  ehgage  them  for  several  years  in  a 
highly  complex  developtnental  effort.    It  was  anticipated  that  the  development 
and  tryout  of  the  new  elementary-middle  school  curriculum  would  require  six 
to  seven  years  before  It  would  be  ready  for  Implementation  nationwide.  During 
this  projected  iv^f'  period  a  great  many  things  would  have  to  be  accomplished. 
A  sophisticated  educational  development  resource  would  have  to  be  chartered, 
staffed,  housed  and  funded.   Money  for  the  project  was  to  be  secured  through 
an  International  loan  agreement  but  It  was  expected  that  the  loan  would  take 
nearly  a  calendar  year  to  negotiate.    If  the  project  was  not  to  be  delayed 
for  a  full  year  while  waiting  for  loan  funds  the  Korean  Government  would  need 
to  proylde  interim  financing  for  project  start-up. 

The  first  concrete  action  which  followed  immediately  the  public  announce- 
ment of  the  planned  reform,  was  the  creation  by  the  Minister  of  Education,  Dr. 
Kwan  Shik  Min,  of  a  small  nucleus  group  to  proceed  with  the  detailed  planning 
efforts.    Professor  Yung  Dug  Lee,  of  Seoul  National  University,  was  designated 
the  chairman  of  this  task  force  and  was  temporarily  detached  from  the  University 
to  the  Ministry  of  Education.    Florida  State  University  was  asked  by  the  Korean 
Government  to  loan  Morgan  to  the  MOE  from  June,  1971  to  September  1972  to  work 

with  the  task  force. 

In  Figure  No.l  is  shown  a  flow  chart  developed  in  1971  by  the  Task  Force 
identifying  the  niajor  ivents  1n  sequence  which  needed  to  be  accomplished  in 
the  first  eighteen  months  of  the  project. 

Temporary  space  and  funding  was  allocated  to  the  Task  Force  and  1n  a 
short  time  it  grew  to  six  senior  professionals  recruited  from  Korean  Universi- 
ties and  research  institutes.    Wtih  a  small  support  staff  the  Task  Force  began 
its  work.    The  FSU  report  was  a  general  statement  of  Korean  educational  problems 
and  relatively  broad-brush  descriptions  of  possible  responses  to  these  problems. 
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It  was  an  impetus  for  refonn  rot  a  detailed  blueprint  for  action. 

The  first  priority  the  Korean  task  force  set  for  Itself  was  the  develop^ 
ment  of  a  complete  plan  for  tha  management  of  the  reform  project.  They 
estimated  the  number  of  Instructional  hours  for  which  materials  of  various 
kinds  would  need  to  be  developed.    For  all  the  subject  areas  for  the  nine 
grade  levels  teaching  materials  for  nearly  8,000  hours  of  instruction  would 
be  required.    This  included  instructional  television,  the  muiti^media  mastery 
learning  units!  texts  and  workbooks,  and  teacher  guides.    Development  groups 
in  other  Korean  research  institutes  had  some  experience  in  these  kind  of 
activities  and  they  assisted  1n  the  calculation  of  the  numbers  of  people  and 
the  different  specialities  that  would  be  required  to  complete  the  work  in  the 
alloted  time  period.    From  this  analysis  It  was  estimated  that  the  development 
group  would  need  to  consist  of  approximately  300  people-^professlonal ,  tech^ 
nical  and  support  personnel.   Among  this  total  complement  of  manpower  a  wide 
range  of  competercies  were  represented.    Writers,  editors,  photogf aphers , 
m  producers  and  technicians,  instructional  designers  were  only  a  few  of 
the  critical  for  which  eKperienced  people  had  to  be  recruited  or  new  people 
had  to  be  trained. 

An  outgrowth  of  this  effort  was  the  beginning  definition  of  the  organl2a= 
tional  structure  of  the  plannid  Korean  Educational  Development  Institute  which 
had  been  described  in  the  FSU  recommendations.    Other  by-products  included 
early  projections  of  office,  laboratory,  studio  and  other  space  requirements, 
as  well  as  the  needs  for  generic  and  specialized  equipment.   The  Program 
Evaluation  and  Review  Technique  (PERT)  was  used  In  this  early  stage  to  show 
how  human  and  material  resources  would  need  to  be  orchestrated  1n  time  to 
insure  the  success  of  the  E^M  project.   Also  revealed  at  this  stage  were  the 
first  projictlons  of  financial  requirements  by  month  and  year  for  the  life  of 
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the  project.    A  reasonably  accurate  estimate  of  cash  flow  and  total  money 
needed  was  essential  for  purposes  of  national  budgeting  and  was  a  prerequisite 
to  the  completion  of  the  International  loan  agreements. 

Parallel  to  this  process  of  man-loading  the  project  was  a  nationwide 
survey  of  human  resources  with  requisite  skills  and  experience  which  could 
bi  employed  to  Work  on  the  project.    A  comparison  of  the  numbers  and  kinds 
of  people  who  were  available  conflmied  the  earlier  judgement  by  the  study 
team  that  twelve  to  eighteen  months  of  training  time  would  be  required  to 
prepare  a  sufficient  staff.   There  were  enough  people  identified  to  begin 
staffing  KEDI,  particularly  in  the  most  senior  professional,  managerial 
positions.    At  this  point  some  of  these  senior  people  were  employed  by  the 
Task  Force  and  the  Task  Force  gradually  began  to  function  as  the  leadership 
staff  of  KEDI-though  the  Institute  had  not  yet  been  Incorporated  by  the 
national  assembly.    Late  in  1971  Dr.  Yung  Dug  Lee  was  selected  by  the 
Minister  of  Education  to  serve  as  the  Director  of  KEDI. 

During  the  last  quarter  of  1971  the  KEDI  staff  focused  on  two  major 
activities.    These  were:    (1)  an  intensive  series  of  meetings,  conferences 
and  presentations  on  the  E-M  project  and  KEDI,  held  with  teachers,  school 
administrators  and  boards,  lay  groups  and  representatives  from  the  public 
media;  and  (2)  the  writing  of  the  International  loan  agreement  In  cooperation 
with  the  Ministry  of  Finance  and  the  Economic  Planning  Bureau.   The  first  of 
these  activities  was  essential  to  broaden  the  base  of  support  for  the  reform 
effort,  respond  to  questions  or  criticisms  and  to  secure  tiie  cooperation  of 
educators  throughout  the  nation.    Very  little  public  disclosure  of  the  nature 
of  the  E-M  project  had  been  made  prior  to  this  time,  though  the  systematic 
publication  of  all  aspects  of  the  project  to  all  levels  of  Korean  society 
has  been  an  important  continuing  process  since  the  early  part  of  1972, 


In  late  February,  1972,  a  series  of  events  occurred  which  threatened  to 
seriously  jeopardize  the  entire  project.    By  this  time  the  loan  agreement 
between  the  U.S.  Government  and  the  Republic  of  Korea  was  In  the  final  stages 
of  negotiation.    The  threat  resulted  from  a  small  group  of  Koreans,  who,  for 
variety  of  reasons  were  opposed  to  the  reform  project.   This  group  gained 
access  to  the  Korean  President,  Chung  Hee  Park,  and  asserted  that  the  project 
represented  a  danger  for  the  Republic  and  would  be  less  effective  and  more 
costly  than  the  existing  system.    As  a  consequence  the  President  ordered  all 
project  actiyities  halted  until  the  project  could  be  thoroughly  studied  by  a 
special  Presidential  Commission  which  he  appointed.    The  Commission,  made  up 
of  leading   citizens,  educators  and  government  officials  who  had  not  been 
previously  involved  In  the  project,  undertook  an  Intensive  review  of  the 
reform  plan  hearing  from  professional  educators,  government  officials  and  any 
other  Koreans  who  could  contribute  to  their  judgement.   These  hearings  lasted 
for  several  weeks,  with  all  points  of  view  being  heard,  before  the  final 
recommendations  were  made  to  President  Park.   The  Cormtsslon  recognized  that 
certain  risks  were  Involved— as  1n  any  major  change  effort— but  they  judged 
the  project  to  be  soundly  conceived  and  that  the  potential  benefits  to  Korea 
far  outweighed  any  associated  dangers.    The  various  anegations  about  the 
refomi  plan  were  judged  to  be  unimportant  or  without  substance  and  the  Presi- 
dent was  advised  to  let  the  work  proceed— which  he  did.   A  salutory  side-effect 
of  this  Interesting  episode  was  that  many  key  people,  Including  the  President 
and  the  Prime  Minister,  had  achieved  a  thorough  understanding  In  all  that  was 
involved  In  the  proposed  reform.   This  depth  of  understanding  by  the  highest 
level  Koreans  was  to  pay  dividends  for  the  project  In  the  years  ahead. 

On  July  31,  1972  the  Korean  National  Assembly  approved  the  international 
loan  agreatient  for  the  educational  development  project  and  one  month  later. 


August  30,  1972  the  Korean  Educational  Development  Institute  was  officially 
registered  as  a  special  foundation.    While  there  were  still  a  few  legal  loose 
ends  to  tie  up,  for  all  intents  and  purposes  an  enduring,  sanctioned  organiza- 
tion for  educational  change  had  been  created.  A  board  of  directors  was  desigriated 
who  quickly  confirmed  Dr.  Yung  Dug  Lee's  appointment  as  KEDI's  first  director. 

While  the  raison  d'etre  for  KEDI— at  least  at  the  beginning— was  the 
conduct  of  the  Elementary-Middle  School  project  It  was  expected  that  It  would 
also  serve  other  purposes  for  the  Ministry  of  Education.    The  Project  Loan 
Proposal  states  four  broad  aims  of  the  Institute. 

KEDI  Purposes; 

1.  To  detemiine  educational  ideals  and  objectives  which  reflect  the 
cultural  heritage,  social  reality  and  future  direction  for  the 
Korean  society. 

2.  To  reformulate  and  systematize  educational  content  to  correspond 
to  educational  objectives. 

3.  To  develop  and  utilize  modern  educational  methods,  facilities  and 
materials  to  achieve  an  effective  and  economically  efficient 
program  of  education. 

4.  To  establish  a  comprehensive  research  and  development  agency  to 
assist  the  Ministry  of  Education  in  formulating  educational  policy 
for  the  nation. 

These  statements  Indicate  a  considerably  broader  purpose  for  KEDI  than  . 
just  the  E-M  project  and  activities  of  the  Institute  since  its  start-up  has 
reflected  these  broader  purposes.    KEDI  has  inaugurated  a  "High  School  of 
the  Air"  program  which  allows  out  of  school  adults  to  continue  their  high 
school  education  through  radio  Instruction  1n  cooperation  with  participating 
local  high  schools.    KEDI  has  also  been  funded  to  develop  special  curriculum 
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in  the  area  of  population  education.    It  has  and  1s  participating  in  several 
other  similar  kinds  of  projects  at  the  request  of  the  MOE  which  are  worthy 
contributions  to  educational  change  but  are  only  peripherally  related  to  the 
E-M  project. 

Still  the  Elemintary-Middle  School  project  1s  the  main  driving  force  of 
KEDI  and  much  work  has  been  completed  on  that  project.    For  the  first  couple 
of  years  the  attention  of  the  KEDI  leadership  had  to  be  focused  on  Institution 
development.    This  took  longer  and  v/as  more  difficult  than  any  of  the  original 
planners  had  anticipated,    KEDI  entered  Into  an  agreement  with  FSU  for  help 
with  their  staff  development  and  for  on-call  technical  assistance.  Clearly 
one  of  the  most  urgent  needs  of  KEDI  was  to  rapidly  train  and  expand  its 
professional  staff. 

Since  1974  the  curriculum  and  instructional  developnient  effort  has  repre- 
sented the  main  activities  of  KEDI  on  the  E41  project.    Prlnci^l  areas  of 
activities  include: 

1.    Development  and  tryout  of  new  instructional  delivery  rnodels 
including  classroom,  media  and  management  considerations* 

2*    Development  of  Instructional  materials  and  programs  (prints  ITV 
and  radio)  compatibla  with  MOE  currlculuni  and  suitable  for  the 
new  instructional  delivery  system. 

3,  Construction  and  operation  of  broadcasting  and  transmitting 
facilities  for  television  and  radio. 

4.  Experimentation  and  evaluation. 

5,  Development  of  teacher-training  programs  and  materlalsi  training 
of  participating  teachers  and  administrators. 

6.  Development  and  maintenance  of  a  network  of  contacts  and  communica- 
tion linkages  at  provincial  and  local  levels* 
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Field   Validation  and  National  Diffusion   -  1978-79 

As  the  project  has  matured  and  the  KEDI  leadership  has  gained  experience 
many  changes  in  the  original  plan  of  action  have  been  made.    The  systematic 
emphasis  of  the  FSU  study  appear,  to  have  survived    but  many  of  the  specific 
strategies  for  implementation  have  been  mod1fied--sometimes  substantially. 
The  E-M  project  called  for  several  major  changes  in  Korea's  elcrnentary  and  middle 
programs.    While  the  strongest  rationale  in  1971-72  for  the  reform  was  in  the 
cost  area,  this  was  soon  reduced  as  a  priority  by  KEDI-  and  the  Ministry  of 
Education.    One  reason  for  this  shift  was  the  Korean  economy  and  its  growth 
during  this  period.    The  FSU  report  had  projected  a  continuing  shortfall  in 
the  possible  school  enrollments  based  on  Korea's  predictions  in  growth  rate 
of  its  GNP  from  1970  to  1978,  assuming  that  the  share  of  GNP  allocated  education 
would  remain  relatively  constant.    The  report  also  anticipated  a  continuing 
shortage  of  teachers.    Both  of  the  predictions  missed  the  mark.    GNP  grew  at 
a  much  higher  than  predicted  rate  and  more  teschers  were  trained  and  available 
for  classroom  assignment  than  had  been  expected  because  of  substantial  increases 
1n  salaries.    With  a  fairly  large  Increase  in  per  capita  GNP  there  was  simply 
more  absolute  money  available  to  support  education^  even  though  the  relative 
amount  remained  about  the  same, 

There  are  two  consequences  of  this:    (1)  Korea  has  been  able  to  enroll 
nearly  90  percent  of  the  age  cohort  1n  the  first  nine  years  of  education  and 
expect  to  make  attendance  for  elementary  middle  school  education  compulsory 
and  available  for  all  In  the  early  1980s.    (2)  KEDI,  relieved  In  large  measure 
of  the  numbers  problem^  could  concentrate  on  the  E-M  project  in  Improving 
instructional  efficiency  and  content  relevance.    With  the  reduced  pressure  to 
save  money,  some  of  the  more  politically  and  socially  troublesome  recommendations 


of  the  FSU  report  were  abandoned  mrl^  in  the  project*    These  included  double- 
shifting  of  schools,  differentiated  teaching  staff  functions,  and  grouping 
students  into  larger  learning  groups.    ITV  and  instructional  radio  have  also 
ekperienced  a  major  change  in  their ^respective  roles  in  instructional  delivery, 
but  for  different  reasons.    It  was  originally  intended  that  they  would  be  an 
Integral  part  of  the  daily  Instruction  of  all  students,  and  Indeeds  the  Instruc- 
tion dGllvered  by  ITV  and  radio  would  not  be  given  any  other  way*    In  other 
words,  ITV  and  radio  were  to  have  been  a  critical  part  of  the  Instructional 
delivery  system,    ThesQ  elements  had  been  includeU  in  Lhe  proposed  instructional 
repertoire  to  contribute  both  to  cost-saving  and  Improvement  In  the  quality 
of  Instruction. 

The  Korean  government  entered  Into  a  contract  with  the  Tethered  Communication 
Corporation  (T-Coni)  of  Westinghouse  In  December^  1972  for  the  purchase  of  a  TV 
and  radio  broadcast  transmission  system  to  be  installed  in  the  north  central 
part  of  the  country,    KEDI  built  a  27,050  square  meter  facility  to  house  and 
support  the  T-Com  transmitting  operation*    The  T-Com  transmitting  system 
included  a  helium  filled  tethered  balloon  flying  at  an  altitude  of  10,000  feet 
to  which  transmitters  are  affixed.    It  was  planned  for  KEDI's  ITV  and  radio 
programs  to  originate  at  the  main  studios  near  Seoul i  be  relayed  to  the  T-Com 
transmitters  carried  by  the  balloon  on  a  C  Band  nircrowave  link  and  be  retrans- 
mitted to  television  and  radio  receivers  In  school  classrooms*    Two  UHF 
channels  were  dedicated  for  ITV  and  one  FM  channel  for  radio.    This  first 
site  was  to  have  been  one  of  a  two  phase  operation.    The  second  pha^e  would 
have  seen  a  second  balloon  Installed  in  the  southern  part  of  the  Republic  and 
the  two  sites  together  would  have  distributed  an  acceptable  signal  to  the  entire 
country.  Including  the  off-shore  islands. 


This  aspect  of  KEDI's  project  was  fraught  with  technical  problems  from 
the  onset,  both  in  terms  of  the  aerodynafnic  stability  of  the  balloon  and  the 
quality  of  the  signaU    After  a  few  trial  bfoadcasts  In  late  1975,  the  technical 
problems  were  Judged  insurmountable  in  the  Korean  climate.    The  system  was 
declared  unworkable  and  the  site  was  dismantled  in  1977  (Masoner^  1979). 

During  the  same  period  KEDI  had  completed  the  Installation  of  its  TV  and 
radio  production  studios  and  many  lessons  using  these  media  had  been  prDduced. 
KEDI  made  the  decision  to  develop  a  groundbased  microwave  relay  system  when 
the  T-Com  technology  was  abandoned.    Such  a  system  would  parallel  the  existing 
microwave  relay  system  presently  owned  and  operated  by  the  Korean  Broadcasting 
System  (KBS)  under  the  Jurisdiction  of  the  Ministry  of  Communications.  In 
early  1979,  discussion  v/ere  still  underway  in  Korea  as  to  whether  responsibility 
for  the  educational  transmission  network  would  remain  with  KEDI  and  the  MOE  or 
be  assigned  to  KBS  and  the  Ministry  of  Communication.    However   this  decision 
goes,  it  seems  certain  that  there  will  be  a  nationwide  broadcast  network  dedi- 
cated solely  to  educational  programs,  and  that  KEDI  will  retain  programrningt 
production  and  scheduling  control,  even  if  it  does  not  own  and  operate  the 
transmitter  system. 

While  the  T-Com  Corporation  iridemnified  Korea  for  its  financial  losses  in 
this  venturep  the  time  lost  by  KEDI  cannot  be  replaced,  nor  can  the  credibility 
it  lost  with  the  Korean  public*  as  a  result  of  promising  national  color  tele- 
vision broadcasts  which  it  has  been  unable  to  deliver* 

Because  of  the  failure  of  the  tethered  balloon  transmission  system  bought 
from  Westinghouse  It  was  impossible  for  KEDI  to  build  a  reliance  on  the  ITV  and 
radio  components  during  the  devleopment  period.    The  instruction  to  have  been 
provided  by  ITV  and  radio  had  to  be  given  in  other  ways,  using  teachers  and 


instructional  materials  which  could  be  made  available  to  remote  schools.  While 
a  great  many  instructional  programs  in  ITV  and  radio  were  developed  by  KEDI-- 
and  most  of  these  were  tried  out  in  classrooms  on  a  fragmental  basis— the  new 
instructional  system  neither  depended'upon  them  nor  provided  for  their  inclusion 
as  an  integral  instructional  element.    It  now  appears  these  programs  will  be 
broadcast  as  adjunct  to  the  new  teaching  system,  rather  than  as  an  essential 
part  of  it. 

What  then  remains  of  the  ambitious  reform  project  started  in  1971?  As 
it  turns  out,  a  great  deal  remains  and  probably  all  of  the  truly  essential 
concepts  have  been  preserved.    KEDI  seriously  addressed  the  task  of  analyzing 
the  existing  curriculum  and  the  educational  needs  of  Korea's  citizenry  and 
systeniaticany  identified  the  places  where  the  curriculum  was  not  sufficiently 
responsive  to  these  needs.    It  then  developed  relatively  Inexpensive  instructional 
materials  and  student  learning  evaluation  instruments  which  articulated  with 
those  text  and  other  materials  already  in  common  use.    It  devised  a  teaching/ 
learning  process  which  could  be  managed  by  the  existing  teaching  personnel  and 
school  administrators.    It  then  devised  teaching  guides  and  intensive  training 
programs  for  teachers  and  administrators  to  prepare  them  to  function  effectively 
as  the  orchestrators  of  the  new  instructional  programs. 

Finally,  they  conducted  an  Iterative  series  of  validation  tryouts  of  the 
new  instructional  elements  and  the  elements  1n  aggregation.    These  field  tryouts 
were  started  In  May,  1973  and  only  745  students  from  two  schools  1n  Seoul  were 
Involved.    There  were  two  subject  areas—mathematics  and  science— in  the  third 
and  fifth  grades  which  were  tested.    This  test,  the  first  of  several,  was  the 
means  of  formative  evaluation  and  feedback  of  the  new  instruction.    It  told  the 
KEDI  developers  what  needed  to  be  changed  in  the  materials  and  processes  tested 


and  provided  a  basis  of  experience  for  future  development.   The  indicated  changes 
were  on  the  basis  of  how  effectively  the  programs  caused  children  to  learn.  It 
was  the  beginning  of  a  competency  based  program  of  student  learning. 

During  the  successive  demonstration  tryouts  KEDI  learned  a  great  deal 
about  areas  where  remediation  materlafs  were  needed  for  some  students  and 
enrichment  programs  were  needed  for  others.    Student  learning  data  suggested 
areas  for  individualized  or  sel f- teaching  and  pointed  up  those  areas  where 
teacher  directed  group  instruction  was  most  appropriate.    Most  Important^  the 
teachers  and  students  know,  on  a  daily  basis  what  the  desired  learning  outcomes 
were  and  had  a  means  of  assessing  progress  toward  these  ends.    The  latest 
tryout,  completed  1n  1978,  Included  231,567  students  in  the  first  six  grades  of 
231  schools  from  throughout  the  nation.    All  subject  areas  were  Included. 
Student  achievement  data,  student,  parent  and  teacher  reaction,  and  process 
efficiency  data  were  collected  from  both  the  tryout  schools  and  a  comparable 
sarnple  of  control  schools  providing  conventional  instruction.    As  of  the 
writing  of  this  paper  the  massive  array  of  data  collected  during  this  tryout 
has  not  been  completely  analyzed  but  the  results  on  all  of  the  major  variables 
significantly  favor  the  new  program.    In  the  several  tryouts-four  small  scale 
and  four  large— since  1973  the  achievement  levels  have  generany  been  higher 
for  the  demonstration  students  than  for  the  comparison  group.    As  a  rule,  the 
achievement  differences  between  the  two  groups  tended  to  become  larger  with 
each  successive  tryout. 

In  1978,  the  President  of  the  Republic  appointed  an  external  coirmlsslon 
to  conduct  an  Independent  evaluation  study  of  the  new  E-M  program.    This  group 
assessed  student  and  teacher  attitudes  toward  the  new  program  as  well  as  community 
reaction.    They  also  selected  18  schools  with  some  3500  students  enrolled,  and 
directed  that  the  new  KEDI  system  be  Implemented  In  these  schools  for  five 
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months  in  six  basic  subject  areas,  and  identified  a  group  of  traditional  schools 
to  serve  as  the  control.    They  found  that  mean  achievement  across  all  of  the 
subject  areas  was  24  percent  higher  in  the  e?<per1  mental  group  than  in  the  control 
group,  and  that  30  percent  more  of  the  experimental  students  achieved  subject 
mastery.    Another  finding   was  that  experimental  students  1n  rural  schools- 
rural  students  were  usually  well  behind  urban  students  1n  ach1evement"performed 
as  well  as  students  enrolled  in  city  schools.    The  commission  recommended  an 
orderly  implementation  of  the  new  E-M  program  in  all  of  Korea's  schools. 
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KEDI  has  produced  a  large  number  of  research  reports,  monographs  and 
position  papers  which  have  much  potential  use  for  professional  educators. 
Many  of  these  are  available  In  English  and  may  be  obtained  directly  from  KEDI, 


